Evaluation of an Undergraduate Research Experience

8/12/2019 Evaluation of an Undergraduate Research Experience

1/11

Journal of Educational Psychology Copyright 2000 by the American PsychologicalAssociation, Inc.2000, Vol. 92, No. 1,191-201 0022-0663/00/ 5.00 DOI: 10.1037//0022-0663.92.1.191

Evaluat ion o f an Undergraduate R esearch Exp er iencePercep t ions o f Undergraduate Interns and Th eir Facu lty Men torsCarolAnne M Kardash

U n i v e r s it y o f M i s s o u r i - - -C o l u m b i aThis study evaluated the extent to which 14 research skil ls were enhanced by scienceundergraduates' participation in an undergraduate research experience (URE). Fifty-sevenundergraduates self-rated their abili ty to perfo rm the skil ls at the beginning and end of theURE. Faculty m ento rs ' rat ings of their respective interns' skil ls served as an ob jective measureof intern skil l level. Mentor and intern data revealed that the UR E enhanced som e skil ls betterthan others. At the end o f the U RE, fem ale interns rated their abili ty to understand concepts intheir f ield significantly low er than did male interns. Fem ale interns also tended to perceive lessof an increase in their ability to formulate research hypotheses than did male interns.

T e n y e a r s a g o , a r e p o r t f r o m t h e N a t i o n a l S c i e n c eF ou nda t i o n ( NS F ; 1989) s t a ted , I t i s c l ea r t ha t t he aca -d e m i c c o m m u n i t y r e g a r d s t h e i n v o l v e m e n t o f u n d e r g r a du a t es t u d e n t m a j o r s i n m e a n i n g f u l r e s e a r c h . . . w i t h f a c u l t ym e m b e r s a s o n e o f t h e m o s t p o w e r f u l o f in s tr u c ti o n a l t o o l s( p . 6) . S i n c e t h en , t h e e n g a g e m e n t o f u n d e r g r a d u a t e s i n s u c hr e s e a r c h i n t e r n s h i p s h a s b e c o m e a n i n c r e a s i n g l y i m p o r t a n tc o m p o n e n t o f s c ie n c e c u r r ic u l a . A c c o r d i n g t o H a k i m ( 1 9 9 8 ) ,u n d e r g r a d u a t e r e s e a r c h e x p e r i e n c e s ( U R E s ) a r e c h a r a c t e r -i zed by f our f ea t u r es : ment or sh i p , o r i g i na l i t y , accep t ab i l i t y ,a n d d i s s e m i n a t i o n . H a k i m n o t e d f u r t h e r t h a t t h e s e f o u rf e a t u r e s r e s t o n t h e f o l l o w i n g a s s u m p t i o n s : F i r s t , U R E sa s s u m e a n i n t e r a c t i o n b e t w e e n t h e u n d e r g r a d u a t e r e s e a r c hi n t e r n a n d f a c u l t y m e n t o r t h a t i s f o c u s e d o n t h e s t u d e n t ' sl e a r n in g . S e c o n d , t h e U R E i s e x p e c t e d t o l e a d t o a m e a n i n g -f u l c o n t r i b u t io n b y t h e u n d e r g r a d u a t e t o t h e r e s e a r c h p r o j e c t.T h i r d , t h e p r o c e d u r e s a n d m e t h o d s u s e d a s t o o l s o f i n q u i r yb y t h e u n d e r g r a d u a t e a r e c o n s i s t e n t w i t h c u r r e n t p r a c t i c e s int h e d i s c ip l in e . F o u r t h , t h e U R E i s e x p e c t e d t o c u l m i n a t e i n at a n g i b l e p r o d u c t t h a t i s c r i ti q u e d b y o t h e r m e m b e r s o f th ed i sc i p l i ne .D r i v e n b y t h e n e e d f o r a c c o u n t a b i l i t y t o t h e e x t e r n a lf u n d i n g a g e n c i e s t h a t s u b s i d i z e U R E s i n t h e s c i e n c e s , a sw e l l a s b y t h e n e e d t o j u s t i f y t h e l a r g e a m o u n t o f f a c u l t y a n ds t u d e n t t i m e r e q u i r e d b y t h e s e i n t e r n s h ip s , a s s e s s m e n t o f t h ee f fi c a cy o f U R E s h a s t a k e n o n m a j o r i m p o r ta n c e . O u t c o m e so f U R E s h a v e t y p i c a l ly b e e n a s s e s se d b y t r ac k i n g t h en u m b e r o f i n t e rn s w h o s u b s e q u e n t l y p u r s u e g r a d u a t e d e -g r e e s i n s c i e n c e , t h e n u m b e r o f c o n f e r e n c e p r e s e n t a t i o n s a n dp u b l i c a t i o n s o n w h i c h i n t e r n s s e r v e a s c o a u t h o r s , a n d b y

This research was sponsored by the National Science FoundationRecognit ion Aw ard for the Integration of Research an d Education.Portions of this art icle we re presented at the annual m eeting of theAm erican Educational Research Association, Montreal , Canada,April 1999.Correspondence concerning this art icle should be addressed toCarolAnne M . Kardash, Department of Educational and Counsel-ing Psychology, 16 Hill Hall , Universi ty of Missouri , Columbia,Missouri 65 21 1. Electronic m ail may be sent to [email protected],du.

r e t r o s p e c t i v e r e p o r t s f r o m i n t e r n a l u m n i ( K r e m e r & B r i n g l e ,1 9 9 0 ) . T h e s e m e a s u r e s a r e u n d o u b t e d l y t h e m o s t i m p o r t a n ti n d i c a t o r s o f t h e l o n g - t e r m s u c c e s s o f s c i e n c e U R E s .H o w e v e r , t h e y a r e l e s s h e l p f u l f o r a s s e s s i n g t h e e x t e n t o fl ea r n i ng t ha t t akes p l ace whi l e t he s t uden t i n t e r ns a r ea c t u a l l y e n g a g e d i n t h e r e s e a r c h e x p e r i e n c e ( B l o c k u s ,K a r d a s h , B l a i r , & W a l l a c e , 1 9 9 7 ) , o r s t u d e n t s ' p e r c e p t i o n s o fw h a t t h e y l e a r n e d a s a r e s u l t o f th e i r U R E s .

T h e r e i s c o n s i d e r a b l e c o n s e n s u s a m o n g p r o g r a m d i r e c t o r sa n d f a c u l t y m e n t o r s r e g a r d i n g t h e o u t c o m e s t h a t t h e y e x p e c ts t u d e n t s w i l l a c q u i r e a n d d e m o n s t r a t e u p o n c o m p l e t i o n o fU R E s . P r o b a b l y t h e m o s t o f t e n c it e d o u t c o m e i s th e a b i l i tyt o d o s c i e n c e . T h i s a b i li t y i s t y p i c a l l y d e f i n e d a s u n d e r -s t a n d i n g a r e s e a r c h p r o b l e m i n s u f f ic i e n t d e p t h s o a s t o b ea b l e to p o s e a q u e s t i o n a b o u t it , d e t e r m i n i n g w h a t e v i d e n c ei s n e e d e d t o s o l v e t h e p r o b l e m , a n d c o l l e c t in g t h e d a t a t h a tw i l l a n s w e r t h e q u e s ti o n ( M a n d u c a , 1 9 9 7 ). O t h e r f r e q u e n t l yv o i c e d o u t c o m e s i n c l u d e t h e a c q u i s i t io n o f k n o w l e d g e ,r e sea r ch sk i l ls , and t he a t t it udes o f s c ien t i s t s ( Ahl m, 1997) ;t h e a b i l i t y t o t h i n k i n d e p e n d e n t l y ( A h l m , 1 9 9 7 ; M a n d u c a ,1997) ; g r o wt h i n o r i g i na l i t y , c r ea t i v i ty , i n i t i a ti ve , cu r i os i t y ,e n t h u s i a s m , a n d r e s o u r c e f u l n e s s ; th e a b i l i ty t o c o m m u n i c a t ei d e a s ; a n u n d e r s ta n d i n g o f t h e o r y a n d p r o c e d u r e s ; k n o w l -edg e o f pe r t i nen t l i t e r a t u r e ; and adep t nes s i n t he f i e l d o rl abor a t o r y ( Dav i s & Gl az i e r , 1997) .

M u c h l e s s c o n s e n s u s e x i s ts c o n c e r n i n g t h e s p e c i f i c c r it e -r i a th a t a r e u s e d t o a s s e s s t h e s e a n t i c i p a te d o u t c o m e s ( D a v i s& G l a z i e r, 1 9 9 7 ). M e a s u r i n g t h e q u a l i ty o f e a c h i n t e r n ' sr e s e a r c h e x p e r i e n c e w i t h a n y s o r t o f s ta n d a r d i z e d m e a s u r e i sm a d e e s p e c i a l l y d i f fi c u lt b e c a u s e s t u d e n t s e n t e r U R E s w i t h av a r i e t y o f b a c k g r o u n d s , r e c e i v e g u i d a n c e f r o m f a c u l t ym e n t o r s w i t h d i f f e r e n t s t y l e s a n d g o a l s , a n d c o n d u c t r e -sea r ch i n d i f f e r en t d i s c i p l i nes and a t d i f f e r en t l eve l s o ft echn i ca l soph i s t i ca t i on ( B l ockus e t a l . , 1997) . T hus , mos ta t t e m p t s t o m e a s u r e o u t c o m e s t o d a t e h a v e r e l i e d p r i m a r i l yo n a n e c d o t a l e v i d e n c e . T h e s e d a t a r e v e a l t h a t s t u d e n ts e n j o yt he a t t en t i on o f facu l t y , t he i n t e l lec t ua l cha l l eng e o f re -s e a r c h , a n d t h e c a r e e r a d v a n t a g e s t h a t f a c u l t y m e n t o r i n gc o n f e r s ( c f . K a r d a s h , 1 9 9 9 ) . F a c u l t y m e n t o r s , o n t h e o t h e rh a n d , e n j o y t h e o p p o r t u n i t i e s t o w o r k c l o s e l y w i t h b r i g h t ,

191


2/11

192 KARDASHenerge t i c unde rgradu a tes , and to s ee the i r theor ie s t e s ted inthe l abora to ry (Hakim , 1998) . S t i ll , a re UR Es jus t a fee lg o o d e x p e r i e n c e , o r d o t h e y ac t u a l ly e n h a n c e s tu d e n t ' sin te l l ec tua l ach ievem ent (S p i l ich , 1997 , p . 57)?

T h e o r e t i c a l F r a m e w o r kM u c h t h e o r e t i c a l s u p p o r t c a n b e m a r s h a l e d t o s u p p o r t t h e

c o n t e n t i o n t h a t U R E s s h o u l d p r o v i d e a p a r t ic u l a r ly e f f e c t i v em eans fo r enhanc ing s tuden t s ' in te l l ec tua l g rowth . In fac t ,U R E s e p i t o m i z e w h a t c o n s t r u c ti v i s t e d u c a t o r s t e r m cognitive apprenticeship m ode ls o f ins t ruc t ion , wh ich , in tu rn , a reg r o u n d e d i n s i tu a ted co g n i t io n m o d e l s o f l e a rn i n g . P r o p o -n e n t s o f s i tu a t e d c o g n i t i o n a r g u e t h a t c o n c e p t u a l k n o w l e d g ecannot be abs t rac ted f rom the s i tua t ions in which i t i sl ea rned and used . Ra the r , knowledge i s cons ide red to be as e t o f c o n c e p t u a l t o o l s t h a t i s b e s t u n d e r s t o o d th r o u g h it suse in the au then t ic ac t iv i t ie s o f a pa r t i cu la r dom ain (B rown,Col l ins , & Duguid , 1989; Lave , 1995) . The no t ion i s tha tm e a n i n g f u l , c o n c e p t u a l l e a rn i n g i s b e s t p r o m o t e d b y c o g n i -t ive appren t i cesh ips in which novices in a d i s c ip l ine workin tens ive ly wi th expe r t s to accom pl i sh re levan t t a sks .T h r o u g h t h e s e a p p r e n t ic e s h i p s, n o v i c e s l e a r n n o t o n l y h o wto pe r form the t a sk , bu t a l so to th ink about the t a sk in thesam e w ay as do expe r t s in tha t dom ain (Lave , 1995) .

S i t u at e d c o g n i t i o n a n d c o g n i t i v e a p p r e n t ic e s h i p m o d e l s o flea rn ing have a t the i r core the fo l lowing as sum pt ions : (a )Lea rn ing i s a soc ia l p roces s ; (b ) com pe ten ce in a dom ain i sde f ined in t e rm s of expe r t i s e ra the r than inna te ab i l i ty ; (c )m e a n i n g f u l l e a r n i n g i s a c t i v e , c o n s t r u c t i v e , a n d s e l f -regu la t ing ; and (d ) l ea rn ing ac t iv i t i e s shou ld re f l ec t rea l -wor ld ra the r than decontex tua l i zed academ ic t a sks (S hue l l ,1997 , p . 751) . C lea r ly , UR Es a ccord we l l wi th these m ode l s 'v i e w s o f t h e l e a r n e r a n d t h e l e a r n i n g p r o c e s s . T h r o u g h aproces s o f encu l tu ra t ion (Brown e t a l . , 1989) , in tem s whop a r t i ci p a t e in U R E s a r e g i v e n t h e o p p o r t u n i t y to o b s e r v e a n dm ode l re sea rch sk i l l s dem ons t ra ted by facu l ty m entors , tom o d i f y s k i l l s o n t h e b a s i s o f f e e d b a c k f r o m m e n t o r s a n dp e e r s , t o r e c e i v e r e i n f o r c e m e n t f o r s u c c e ss f u l e n a c t m e n t o fsk i l l s , and to l ea rn to ac t in accord wi th the norm s tha tgov ern the p rac t i ce o f s c ien t i fi c re sea rch .

U R E s a r e a l s o c h a r a c t e r i z e d b y a n u m b e r o f t h e s a m efea tures tha t educa t iona l re sea rche rs specu la te a re a s soc i -a ted wi th the m o s t e f fec t ive types o f au then t ic ac t iv i t i e s.F i r s t , UREs requ i re tha t s tuden t s have som e pre requ i s i t eb a c k g r o u n d k n o w l e d g e o f t h e d o m a i n o r t o p i c u n d e r i n v es t i-ga t ion . S econ d , the sk i l l s t aught a re those tha t requ i re h ig he rorde r th ink ing such a s fo rm ing and t e s t ing hypotheses ,s y n t h e s iz i n g i n f o r m a t i o n , a n d s o l v i n g p r o b l e m s ( R i c h m o n d ,1998) . Th i rd , the sk i l l s and ac t iv i t i e s requ i re tha t s tuden t sseek ou t in form a t ion and in tegra te in form a t ion ac ros s d i s c i -p l ines . F our th , s tuden t s a re encourag ed to s e t h igh s tanda rdsf o r p e r f o r m a n c e b u t a t t h e s a m e t i m e t o t a k e r i s k s a n dexper im ent wi th new s t ra teg ie s . F ina l ly , the ou tcom e of thea u t h e n ti c e n d e a v o r i s c o m p l e x a n d u n p r e d i c t a b le ( N e w m a n n& W eha lage , and P ad s & TUrner , a s c i t ed in Orm rod , 1999) .

R e s e a r c h Q u e s t i o n sD e s p i t e t h e b u r g e o n i n g n u m b e r o f U R E s t h a t a r e m a d eava i l ab le fo r s c ience undergradua tes , l i t t l e i s known aboutt h e s u c c e s s o f t h e s e p r o g r a m s b e c a u s e o f t h e l a c k o fem pi r ica l educa t iona l re sea rch on them (Koba l la , But t s , &

Ri ley , 1995) . The purpose o f th i s s tudy was to deve lop aquan t i f i ab le l is t o f re sea rch sk i l l s tha t can be u sed subse -quen t ly to a s ses s the degree to which those sk i l l s a ree n h a n c e d b y p a r t ic i p a t io n i n a U R E . U n d e r g r a d u a t e r e s e a r c hin te rns s e l f - ra ted the i r ab i l i ty to pe r f orm each of the re sea rchs k il ls a t b o t h t h e b e g i n n i n g a n d e n d o f U R E s t h a t t o o k p l a cedur ing e i the r the 19 97-98 academ ic yea r o r the 1998sum m er s em es te r . F acu l ty m entors ' ra t ings o f the i r re spec -t ive re sea rch in te rns ' ab i l i ty to pe r fo rm those s am e sk i ll s a tt h e e n d o f t h e U R E w e r e u s e d a s a n o b j e c t i v e m e a s u r e o fintern sk i l l level .The fo l low ing spec i f i c re sea rch ques t ions w ere o f in te re s t:1 . Wha t re sea rch sk i l l s do undergradua tes m os t hope tod e v e l o p b y t h e i r p a r t ic i p a ti o n i n t h e U R E , a n d t o w h a t e x t e n td o t h e y b e l i e v e t h a t t h o s e s k i l l s w e r e d e v e l o p e d b y t h e i rpa r t i c ipa t ion? The ra t iona le under ly ing th i s ques t ion i s tha ts tuden t s ' pe rcep t ions about wha t was (o r was no t ) l ea rnedd u r i n g t h e U R E , r e g a r d l e s s o f w h e t h e r t h o s e p e r c e p t i o n sc o r r e s p o n d t o w h a t w a s a c t u a ll y l e a r n e d a n d a c c o m p l i s h e d ,m ay in f luence the i r fu tu re cho ices about pa r t i c ipa t ion inadd i t iona l UREs or o the r ac t iv i t i e s tha t would fu r the rdeve lop the i r re sea rch sk i l l s . Moreover , i t i s im por tan t tok n o w f r o m t h e p e r s p e c t i v e o f t h e r e s e a r c h i n te r n s w h e t h e rthey them se lves a t t r ibu te any grow th in the i r re sea rch sk i ll sspec i f i ca l ly to the i r l ea rn ing expe r ie nces in the UR E.2 . Do re sea rch in te rns ' pe rcep t ions o f the i r ab i l i ty tope r form the re sea rch sk i l l s change a s a re su l t o f the i rpa r t i c ipa t ion in the URE? A s ign i f i can t inc rease in theresea rch in te rns ' p re - to pos t -U RE ra t ings o f the i r ab i li ty top e r f o r m t h e r e s e a r c h s k i l l s w o u l d p r o v i d e o n e s o u r c e o fev idence tha t the URE d id succes s fu l ly enhance in te l l ec tua lgrowth .3 . Do in te rns ' s e l f -eva lua t ions o f the i r l eve l o f re sea rchsk i l l s a t the end of the URE d i f fe r s ign i f i can t ly f rom the i rfacu l ty m entors ' eva lua t ions o f those s am e sk i l l s ? F rom thes tandpoin t o f l ea rn ing theory , l ea rne rs ' ab i l i ty to accura te lyas ses s the i r own lea rn ing and pe r form ance i s an im por tan tm e tac ogni t ive sk i l l. L i fe long l ea rn ing requ i re s tha t ind iv idu-a ls a s s u m e m o r e r e s p o n s i b il i ty f o r t h e i r o w n l e a r n i n g , o n ec o m p o n e n t o f w h i c h i n v o l v e s v e r i d i c a l as s e s sm e n t o f t h e i ro w n p e r f o r m a n c e a n d p r o g r e s s ( F a l c h i k o v & B o u d , 1 9 8 9 ).Mo reover , accura te s e l f -m oni to r ing i s one cha rac te r i s t i c tha td i s ti n g u i sh e s n o v i c e f r o m e x p e r t p e r f o r m a n c e i n a d o m a i n(Gagne , Yekovich , & Yekovich , 1993) . F rom a p rac t i ca ls t andpoin t , s ign i f i can t d i s c repanc ies be tween facu l ty andin te rns ' ra t ings o f in te rn pe r form ance would s igna l a needf o r d e v e l o p m e n t o f e x p l i c i t a v e n u e s f o r f e e d b a c k f r o mm entor to in te rn to be t t e r a s sure tha t in tended l ea rn ingoutcom es a re in fac t ach ieved .4 . Do m a le and fem a le in te rns ra te the i r sk i l l s s im i la r ly ,a n d ( 5 ) d o f a c u l t y m e n t o r s r a t e m a l e a n d f e m a l e i n t e r n s 's k il ls s i m i l a r l y ? T h e q u e s t i o n o f w h e t h e r m a l e a n d f e m a l ein te rns pe rce ive the i r re sea rch sk i l l s s im i la r ly i s im por tan t


3/11

EVALUATIONO F U R E 1 9 3g i v e n p a s t r e s e a r c h t h a t h a s i n d i c a te d t h a t th e u n d e r g r a d u a t ey e a r s a r e a c r i t i c a l f i l t e r p o i n t i n t h e m a t h e m a t i c s , s c i e n c e ,e n g i n e e r i n g , a n d t e c h n o l o g y p i p e l i n e , e s p e c i a l ly f o r f e -m a l e s a n d m i n o r i t i es ( B e t z , 1 9 9 4 ; F a r m e r , 1 9 9 7 ; S e y m o u r &H e w i t t , 1 9 9 7 ) .

M u c h r e s e a r c h h a s b e e n c o n d u c t e d o n p r o c e s s e s t h a ti n f lu e n c e p r e c o l l e g e w o m e n ' s a s p i ra t io n s t o c a r e e r s i ns c i e n c e, m a t h e m a t i c s , e n g i n e e r i n g , a n d t e c h n o l o g y ( S M E T ) .F a c t o r s s u c h a s g e n d e r - b a s e d d i s p a r i t i e s i n c l a s s r o o m i n t e r -a c t io n s , d i ff e re n t ia l e x p e c t a t i o n s f o r m e n a n d w o m e n i nm a t h a n d s c i e n c e c la s s e s ( K a h l e , 1 9 9 0 ; K a h l e & R e n n i e ,1 9 9 3 ; L e a c h , 1 9 9 5 ), a n d a t e n d e n c y f o r w o m e n t o a t t r ib u t ea c a d e m i c p r o b l e m s i n s c i e n c e a n d m a t h e m a t i c s t o a l a c k o fa b i li t y (T o b i a s , 1 9 9 3 ) w o r k t o d e c r e a se w o m e n ' s c o n f i d e n c ei n t h e i r m a t h a n d s c i e n c e a b i l i t i e s ( E c c l e s , K a c z a l a , &M e e c e , 1 9 8 2 ; S a d k e r & S a d k e r , 1 9 8 5) . T h e c o n s e q u e n c e s o ft h e s e p r o c e s s e s a r e n o t i c e a b l e b y n i n t h g r a d e , a f t e r w h i c hg i rl s t e n d to t a k e f e w e r a d v a n c e d m a t h a n d s c i e n c e c o u r s e st h a n d o b o y s a n d t o d o l e s s w e l l a c a d e m i c a l l y i n t h o s ec o u r s e s t h a t t h e y d o t a k e ( S e y m o u r & H e w i t t , 1 9 9 7 ) . T h ec u m u l a t i v e e f f e c t o f th e s e p r e c o U e g e e x p e r i e n c e s i s th a tw o m e n a r e u n d e r r e p r e s e n t e d i n s c i e n c e a t t h e p o s t s e c o n d a r yl e v e l a n d u lt i m a t e l y c o m p r i s e o n l y a b o u t 1 6 % o f w o r k i n gs c i e n t i st s a n d e n g i n e e r s ( T o b i a s , 1 9 9 0 ) .

U n t i l r ec e n t ly , m u c h l e s s w a s k n o w n a b o u t t h e i m p a c t o fu n d e r g r ad u a t e a n d g r a d u a te e x p e r ie n c e s o n w o m e n w h oa c t u a l l y c h o s e S M E T m a j o r s . W h a t i s k n o w n i s t h a tw o m e n ' s p e r s i s t e n c e r a t e i n t h e s e m a j o r s i s s i g n i f i c a n t l yl o w e r t h a n t h a t o f t h e ir m a l e p e e r s ( F e rr e ir a , 1 9 9 8 ; S e y m o u r& H e w i t t, 1 9 9 7 ). E x p l a n a t i o n s f o r w o m e n ' s g r e a t e r le v e l s o fa t t r it i o n i n c l u d e a c h i l l y c l i m a t e f o r f e m a l e s t u d e n t s ( H a l l& S a n d i e r , 1 9 8 2 ) , a n e m p h a s i s o n W e s t e r n p r a c t i c e s o fs c i e n t i f i c i n q u i r y t h a t d o n o t t a k e i n t o c o n s i d e r a t i o n w o m -e n ' s w a y s o f k n o w i n g a n d l e a r n i n g ( R o s s e r, 1 9 9 0 ), f e m a l es t u d e n t s ' f e e l i n g s o f p s y c h o l o g i c a l a l ie n a t io n , d e p r e s s i o n ,a n d d e c r e a s e d s e l f - c o n f i d e n c e i n t h e i r a b i l i t y t o d o s c i e n c e( F e r r e i ra , 1 9 9 8 ) , a n d p e r c e i v e d h o s t i l i t y f r o m a n d n e g a t i v ej u d g m e n t s o f t h e ir q u a l if i c a ti o n s b y s o m e S M E T f a c u l t y a n dt h e i r m a l e p e e r s ( B r u s h , 1 9 9 1 ; S e y m o u r & H e w i t t , 1 9 9 7 ) .

T w o f i n d i n g s f r o m t h e l i t e r a t u r e o n f e m a l e a t t r i t i o n i nS M E T a r e p a r ti c u l a rl y r e l e v a n t t o t he q u e s t i o n o f w h e t h e rg e n d e r a f f e c t s e i t h e r i n t e r n s ' s e l f - r a t in g s o r m e n t o r s ' r a t i n g so f i n t e r n s ' s k i ll l e v e ls . F i r s t, b y t h e s o p h o m o r e y e a r inc o l l e g e , p r o p o r t i o n a t e l y m o r e w o m e n t h a n m e n t e n d t or e p o r t d i m i n i s h e d l e v e l s o f s e l f - e s t e e m , l e s s s e l f - c o n f i d e n c ei n t h e i r a b i l i t y t o d o s c i e n c e , a n d m o r e c o n c e r n s a b o u t t h e i ra c a d e m i c p r o g r e s s, d e s p it e g o o d a c a d e m i c p e r f o r m a n c e s( A r n o l d , a n d S t r e n t a , E l l io t t , M a i l e r , S c o t t , a n d A d a i r , a sc i t e d i n S e y m o u r & H e w i t t , 1 9 9 7 ) . S e c o n d , f e m a l e S M E Tm a j o r s r e p o r t l it tl e e v i d e n c e o f g e n d e r d i s c r i m i n a t i o n b yS M E T f a c u l t y ( S e y m o u r & H e w i t t, 1 9 9 7 ; S tr e n t a e t a l. , a sc i t e d in S e y m o u r & H e w i t t , 1 9 9 7 ) .

M e t h o dD e s c r i p t i o n o f t h e U R E a n d P a r t i c i p a n t s

Participants in this study were undergrad uate scienc e researchinterns and the i r facul ty mentors a t a midwestern , CarnegieResearch I univers ity . Fun ds f rom the Nat ional Science Foun dat ion

(NSF) and the Howard Hughes Medica l Ins t i tu te have suppor tedUR Es fo r b io logy, b iochemis t ry , chemis try , and physics m ajors a tthis insti tution since 1989. Their prima ry purpose is to prepa restudents f or careers in teaching and research. Students participatein theoretical, f ield, or labo ratory research w ith faculty mentors inany o f these d isc ip l ines dur ing an 8-w eek sum mer sess ion or dur ingthe academ ic year . Data fo r th is a r tic le were col lec ted f rom in ternsand mentors who par tic ipa ted dur ing the 1997-9 8 academ ic yearand the 1998 sum mer semester .Research in terns w ho par tic ipa ted in the academ ic year UREworked 12 hr a week in the i r mento rs ' labora tories for a 32-weekper iod. T hei r UR E culm inated wi th ora l presenta tions of the irresearch projec ts a t a s ta tewide sc ience conference . Researchinterns w ho part ic ipated in the summ er UR E worke d 40 hr perweek in the labora tory for an 8-w eek per iod. In addi t ion , summ erinterns w ere encouraged , but not requi red , to a t tend a var ie ty ofworksh ops focus ing on career -re la ted i ssues . Thei r UR E culmi-nated w i th pos ter presenta t ions of the i r work o n the ins t i tu t ion ' scampus .The U RE f o r t he 1997 - 98 a c a de mi c ye a r c ompr i s e d 19 j un i o ra nd s e n i o r s ci e nc e ma j o rs ; t he URE f o r Summ e r 1998 c ompr i s e d38 juniors an d seniors . O f these 57 in terns, 4 9 (86% ) were funde dby HHMI , a nd 8 ( 14%) we r e f unde d by NSF- Re s e a r c h Expe r i -ences for Undergraduates. Thi r ty- three in terns (58%) w ere wom en,and 24 (42%) w ere m en. These in terns comp rised four e thnic it ies :Caucas ian (77%), Afr ican Amer ican (9%), Asian Paci f ic I s lander(11%), and in ternat ional (2%). The sam ple of facul ty mentorscom prised 13 wom en (30%) and 32 men (71%). Ten of the facul tymentors had 2 in terns w orking in thei r labora tory , an d 1 men tor had3 interns in his lab.M a t e r i a l s

The research skil ls to be assessed were chosen init ially on thebas is of a review of the l i te rature ca l l ing for cur r icular reform inundergraduate sc ience (cf . Consor t ium for Pol icy Research inEducat ion , 1995; NSE 1996; Richmond, 1998; Seago, 1992) andon pub l i s he d r e po r t s o f a ne c do t a l ou t c ome da t a g l e a ne d f r omundergraduates wh o had part ic ipated in URE s a t a var ie ty ofinstitutions (Ah lm, 1997; Davis & Glazier, 1997; Krem er &Bringle, 1990; M andu ca, 1997). Man y of these latter reportsappeared in the Decem ber 1997 issue of the Council or Under-graduate Re search Quarterly, which w as devoted to address ing theques t ion of how U REs sho uld be assessed and evaluated . Inaddit ion , s ix facul ty members f ro m the physica l and l i fe sc iences a tthe univers i ty who ha d served rout ine ly as URE m entors for severa lyears were asked to subjec t ive ly evaluate the qual i ty of pos terspresented by undergraduates wh o had part ic ipated in the Summ er1997 URE , and we re asked to d iscuss the specif ic c r ite ria they usedin ma king th eir evaluations.Ant ic ipa ted learning outcomes for UREs, which emerged f romthese three da ta sources we re examined fo r recurr ing responses andthemes , and pre l iminary ca tegor ies were formed on tha t bas is .From those ca tegor ies , a l is t of research ski ll s w as compi led . Thisl is t was then presented to the s ix facul ty mentors w ho had ra ted theposters and to f ive o ther sc ience facul ty members who hadprevious ly served or were current ly serving as URE mentors toreach consensus about the research ski l l s tha t facul ty mentorsexpected in terns to acqui re dur ing the URE. The 14 skil ls v iew edby m entors as mos t impor tant and cr it ica l appear in Table 1 . Eachskil l was ra ted on a 5-po int sca le ranging f rom 1 not at all) to 5 (agreat deal).Althoug h the pr imary in terest in th is s tudy w as on in terns ' andmen tors ' ra t ings of each of the individual 14 research ski l ls ,informat ion w as col lec ted on the in ternal cons is tency of the overa l l


4/11

194 KARDASHTable 1Descriptive Statistics and Paired t Tests or Interns Pre- to Post-URE Self-Ratings and Matched Intern-Mentor Ratingsof Interns Research Skills

Inte rns' self-ratings Mentors' ratings,Beginning of URE End of URE end of UREItem M SD M SD t(56) a M SD t(56) b

Understand contemporary concepts in your field 3.19 0.62 3.82 0.83 5.12 * 3.74 0.79 0.56Make use of the primary scientific research literaturein your field (e.g .,journa l articles) 3.11 0.89 3.68 0.95 4.18 * 3.26 0.94 2.32Identify a specific question for investigation basedon the research in your field 2.85 0.83 3.30 0.88 3.09* 3.31 1.00 -0 .0 2Formulate a research hypothesis based on a specificques tion 2.85 0.82 3.50 0.80 5.23** 3.42 0.94 0.49Design an experiment or theoretical test of thehypothesis 2.64 0.95 3.27 0.93 4.14 * 3.48 1. 03 -1 .0 5Understand the importance of controls in research 4.24 0.61 4.51 0.73 2.44 4.02 0.74 3.62*Observe and collect data 4.02 0.79 4.46 0.73 3.62* 4.42 0.68 0.33Statistically analyze data 3.13 1.04 3.74 0.97 3.79** 3.29 1.15 2.80* (df = 34)Interpret data by relating results to the originalhypothesis 3.00 0.73 3.93 0.78 6.39** 3.85 0.82 0.57Reformulate your orig inal research hypothesis (asappropriate) 2.96 0.71 3.60 0.86 4.45** 3.41 0.98 1.15 (df= 50)Relate results to the bigger picture in your field 2.91 0.86 3.75 0.97 5.24** 3.19 1.02 2.89*Orally communicate the results of research projects 3.07 0.81 3.86 0.85 5.27** 3.95 0.85 -0 .5 4Write a research paper for publication 2.29 0.77 2.96 0.98 4.86** 2.84 1.09 0.75 (df = 36)Think independently 4.14 0.81 3.78 0.99 2.31Note. Each item completes the question To what extent do you feel you can ? Items were rated on a 5-point scale ranging from 1(not at all) to 5 (a great deal). URE = undergraduate research experience.at test for differences in interns' pre- to post-URE ratings, bt test for differences in matched inte rn-men tor atings at the end of the URE.*p < .01, two-tailed. **p < .001, two-tailed.

scale composed of these items. Coefficient alphas were calculatedseparately for the interns and mentors. The internal consistency ofthe scale that was based on ra tings from the 57 research interns was.90, and item-total correlations ranged from .49 to .76. The internalconsistency that was based on the ratings obtained from 24 facultymentors who responded to all 14 items was .96, with item-totalcorrelations ranging from .78 to .88.

ProceduresDuring the first week of either the 1997-98 academic year or

1998 summer UREs, al l research interns were asked to indicate onthe 5-point scale mentioned earlier the extent to which theybelieved that they could perform each skill at that time, and theextent to which they hoped the URE would develop each skill.During the last week of their respective UREs, the interns used thesame scale to rate the extent to which they felt capable ofperforming each skill, and the extent to which they believed theinternship had developed each skill. Dur ing the week following theinte rns' departures from their mentors' laboratories, each intern' smentor used the same scale to rate the extent to which he or shebelieved that the intern was able to perform each skill.It is important to note that mentors were not directed to engageinterns in any special tasks as a part of the evaluation effort or tointeract with interns in any ways other than what they wouldnormally do when working collaboratively with undergraduates onresearch projects in their laboratories. It is also important thatinte rns' self-ratings of their skills were not made available to theirrespective faculty mentors.

Res u l ts and D is cus s ionOverview of Data Analyses

Preliminary analyses examined whether any differencesin inte rns' ratings of the research skills could be a ttributed todifferences in demographic characteristics, self-efficacy, orthe semester during which interns participated in the URE.The five primary research questions dealt with pre- topost-URE changes in ratings, differences between interns'and men tors' ratings, and gender differences in i nterns' andmentors' ratings. Given that there was no theoretical basisfor hypothesizing any interactions among the variables oftime, rater, and intern gender, factorial analyses of variance(ANOVAs) were not used. Rather, the five research ques-tions were addressed with a series of independent-sampleand dep enden t-samp le t tests. To control for Type I error, thedecisi on criterion for statistical significance was set at p .10. Asecond chi-square analysi s revealed that the ethnic composi-tion of the intern groups was not systematically associated


5/11


6/11

196 KARDASHski l ls tha t they ho ped to d eve lop . Ra the r , in te rns ra ted a l l 13sk i l ls a s 4 . 19 o r g rea te r , ind ica t ing tha t they app roached theU R E e x p e c t i n g th a t al l s k il ls w o u l d b e e n h a n c e d q u i t e ab i t o r a g r e a t d e a l .

B y t h e e n d o f t h e U R E , i n t e r n s d i f f e r e n t i a t e d s h a r p l ybe tween those sk i l l s tha t they fe l t were enhanced ve rsusthose tha t were no t . At tha t t im e , they ind ica ted tha t the on lysk i l l tha t had been enhan ced qu i te a b i t was the i r ab i l i ty too r a l l y c o m m u n i c a t e t h e r e s u lt s o f t h e i r r e s e a r c h p r o j e c t s.Thr ee o the r sk i l ls were ra ted h ig h ly (3 . 85 or g rea te r ) a s we l l :obse rv ing and co l l ec t ing da ta , re la t ing the i r re su l t s to the

b i g g e r p i c t u r e i n th e i r f ie l d, an d u n d e r s ta n d i n g c o n t e m p o -ra ry concep t s in the i r f i e ld . By con t ra s t , they be l i eved tha tthe sk i l l tha t had been enhan ced l eas t (ve ry l i t tl e ) was the i rab i l i ty to wr i t e a re sea rch pape r fo r pub l ica t ion . S om ewha tsurpr i sing ly , the rem a in ing sk i ll s , a l l o f which w ere ra ted a so n l y s o m e w h a t e n h a n c e d b y p a r t ic i p a ti o n i n t h e U R E ,inc lude sk i l l s tha t speak m os t d i rec t ly to the h ighe r -orde rsk i l l s invo lved in do ing s c ience , nam ely , iden t i fy ing aspec i f i c ques t ion fo r inves t iga t ion , t rans la t ing the ques t ionin to a workab le hypothes i s , des ign ing a theore t i ca l t e s t o ft h e h y p o t h e s i s , a n d r e f o r m u l a t i n g t h e h y p o t h e s i s o n t h ebas i s o f one ' s expe r im enta l re su l t s . These f ind ings a red i s tu rb ing in tha t m any re sea rche rs in s c ience educa t ioncontend tha t the ab i l i ty to pose ques t ions l i e s a t the ve ryhea r t o f the s c ien t i f i c en te rpr i s e and s c ien t if i c th ink ing (Doff& H erscov i tz , 1999; S hode l l , 1995) .

Resu l t s o f dependent -s am ple t t e s t s conf i rm tha t , wi thou texcep t ion , in te rns ' ra t ings o f the degree to which thei n t er n s h ip w a s p e r c e i v e d t o h a v e d e v e l o p e d e a c h s k il l w e r es ign i f i can t ly lower than ra t ings o f in te rns ' in i t i a l expec ta -t ions tha t the in te rnsh ip would deve lop those s am e sk i l l s .T h e d i s c r e p a n c y b e t w e e n p r e - a n d p o s t - in t e r n s h ip r a t i n g sw a s e s p e c i a l l y m a r k e d ( m e a n d i f f e re n c e s e q u a l t o o r g r e a t e rthan 1 . 00) fo r the fo l lowing sk il ls : wr i t e a re sea rch pape r fo rpubl ica t ion , s t a t i s t i ca l ly ana lyze da ta , re form ula te o r ig ina lre sea rch hyp otheses , and in te rpre t da ta .

T h e o b v i o u s q u e s t i o n r a i s e d b y t h e s e d a t a i s w h e t h e rin te rns ' d i f fe ren t i a l ra t ings o f sk i l l s a t the end of the UREc o r r e s p o n d t o d i f f e r i n g em p h a s e s p l a c e d o n t h o s e s k i ll s b ythe i r facu l ty m entors . When the sk i l l s su rvey was con-s t r u c t e d , c o n s e n s u s w a s o b t a i n e d f r o m m e n t o r s r e g a r d i n gthe re sea rch sk i l l s tha t they thought were im por tan t toas sess . At tha t t im e , non e of the m entors ind ica ted tha t som eo f t h e s e s k i l ls w e r e ( o r s h o u l d b e ) a c c o r d e d r e l a ti v e l y m o r ei m p o r t a n c e i n t h e U R E t h a n o t h e r s. T h e r e i s n o d o u b t t h a tone fac to r tha t in f luences wh e the r in te rns op t fo r pa r t i c ipa -t i o n in m o r e t h a n o n e U R E d u r i n g t h e i r u n d e r g ra d u a t e c a r e e rd e p e n d s o n t h e e x t e n t t o w h i c h t h e y b e l i e v e t h e U R E m e e t sthe i r l ea rn ing needs . The pre - to pos td i s c repanc ies in sk i l l sra t ings repor ted in th i s s tudy prov ide a m eans ea r ly in thein te rnsh ip fo r c la r i fy ing and a t t em pt ing to m a tch s tuden t andfacu l ty m entor goa l s and expec ta t ions , a t l ea s t wi th re spec tto the acqu i s i t ion o f pa r t i cu la r re sea rch sk i l ls . Discus s ionsabout these i s sues , in tu rn , m ay lead in te rns to s eek o the rU R E s t h a t p u t m o r e o f a n e m p h a s i s o n t r a i n i n g t h e p a r ti c u l a rsk i l ls tha t ind iv idua l in te rns a re in te re s ted in acqu i r ing .

Interns Self-Ratings of Skill Levels at the Beginningand End of the URETable 1 p resen t s m eans and s tanda rd dev ia t ions fo rin te rns ' s e l f - repor ted ra t ings o f the i r ab i l i ty to pe r form the13 re sea rch sk i l l s a t bo th the beg inn ing and en d of the URE .

As ind ica ted in the f i r s t co lu m n o f pa i red t t e st s , re su l tsr e v e a l t h at , w i t h t h e e x c e p t i o n o f u n d e r s t a n d i n g t h e i m p o r -tance o f con t ro l s (p = . 018) , in te rns ra ted a l l sk i ll s s ign if i -c a n t l y h i g h e r a t t h e e n d o f t h e U R E t h a n t h e y d i d a t t h eb e g i n n i n g . T h e d i s c r e p a n c y b e t w e e n p r e - a n d p o s t - U R Er a t i n g s i s e s p e c i a l l y m a r k e d ( m e a n d i f f e r e n c e s o f 0 . 7 5 o rgrea te r ) fo r the fo l low ing sk i l ls : in te rpre t da ta , re la te re su l t sto the b igge r p ic tu re in the i r f i e ld , and ora l ly com m unica tethe re su l t s o f re sea rch pro jec t s . By con t ra s t , the d i f fe rencesw e r e l e a st p r o n o u n c e d ( m e a n d i f f e r e n c e s o f 0 . 4 5 o r l e s s) f o rthe fo l low ing th ree sk il ls : iden t i fy a ques t ion f o r inves t iga -t ion , obse rve and co l l ec t da ta , and unders tand the im por-tance o f con t ro l s .I t i s in te re s t ing to no te tha t, a t bo th the beg inn ing and endof the UR E, the low es t ra ted sk i ll s were those tha t dea l t wi thiden t i fy ing a ques t ion fo r inves t iga t ion ; fo rm ula t ing , t e s t ing ,and re form ula t ing hypoth eses ; and wr i t ing a re sea rch paper .Mo reover , the sk i l l s o f iden t i fy ing a ques t ion fo r inves t iga -t i o n a n d t e st i n g h y p o t h e s e s w e r e a m o n g t h o s e th a t s h o w e dt h e l e a st g a i n f r o m t h e b e g i n n i n g t o e n d o f t h e U R E . T h e s ef ind ings , in con junc t ion wi th in te rns ' pe rcep t ions rega rd ingwhich spec i f i c sk i l l s were m os t and l eas t enhanced by theURE, sugges t tha t a l though UREs a re c lea r ly succes s fu l inenhan c ing a n um ber o f bas ic s c ien t i f i c sk i ll s , the ev idenc e isl e s s com pe l l ing tha t UREs a re pa r t i cu la r ly succes s fu l a tp rom ot ing the acqu i s i t ion o f the h ighe r o rde r inqu i ry sk i l l stha t unde r l i e the fou nda t ion of c r i t i ca l , s c ien t if i c th ink ing .

Comparison of Mentors and Interns Ratingsof Skill Levels at the End of the URETable 1 a l so p resen t s the m eans and s tanda rd dev ia t ionsfor facu l ty m en tors ' ra t ings o f the i r re spec t ive in te rns ' sk i l ll e v e l s a t t h e e n d o f t h e U R E . O f t h e 4 5 m e n t o r s w h ocom ple ted the ra t ings o f the i r in te rns ' sk i l l s , s eve ra l ind i -ca ted tha t som e o f the 14 re sea rch sk i l l s tha t they wer e a skedto a s ses s wer e no t app l icab le to the in te rns wi th who m theyworked . S pec i f i ca l ly , 20 m entors ind ica ted tha t the sk i l l o fwr i t ing a re sea rch pape r was no t app l icab le to the in te rn o ri n t er n s w i t h w h o m t h e y w o r k e d ; 2 2 m e n t o r s i n d i c a t e d t h at

the i r in te rns had no oppo r tun i ty to s t a t i s t ica l ly ana lyze da ta ;and 6 m entors ind ica ted tha t the in te rns wi th whom theyw o r k e d h a d n o c h a n c e t o r e f o r m u l a t e h y p o t h e se s .A s c a n b e s e e n i n T a b l e 1 , c o m p a r i s o n o f i n t e r n s' a n d t h e i rre spec t ive m entors ' pos t -URE sk i l l s ra t ings revea l s s t r ik ings im i la r i t i e s . In fac t , m entors and in te rns bo th gave the i rh ighes t ra t ings to the s am e f ive sk i l l s : obse rv ing andco l lec t ing da ta , unde rs tand ing the im por tance o f con t ro l s ,in te rpre t ing da ta , o ra l ly com m unica t ing the re su l t s o f re -s ea rch pro jec t s , and th ink ing independent ly (a l l ra t ingsgrea te r than or equa l to 3 . 75) . S im i la r ly , bo th in te rns andm entors ra ted a s low the sk i l l s o f iden t i fy ing a spec i f i cques t ion fo r inves t iga t ion , fo rm ula t ing a hypothes i s , and


7/11

EVALUATION OF ORE 19 7d e s i g n i n g a t e s t o f t h e h y p o t h e s i s ( a l l r a t in g s l e s s t h a n o re q u a l t o 3 . 5 0 ) . M e n t o r s g a v e t h e i r l o w e s t r a t i n g s t o t h o s esk i l l s t ha t i nvo l ved i n t e r ns ' ab i l i t y t o i n t egr a t e t he i r r e sea r chp r o j e c t s w i t h r e l a t e d r e s e a r c h . T h e s e s k i l l s i n c l u d e m a k i n gu s e o f t h e r e s e a r c h l i te r a t u re , r e l a t in g o n e ' s r e s e a r c h r e s u l t st o t he b i g p i c t u r e , and wr i t i ng a r e sea r ch pape r ( a l l r a t i ngsl e s s t han o r eq ua l t o 3 .25) .A s i n d i c a t e d i n th e s e c o n d c o l u m n o f p a i r e d t t e s ts , r e s u l tsr e v e a l t h a t , c o m p a r e d w i t h t h e i r m e n t o r s ' r a t i n g s , i n t e r n so v e r e s t i m a t e d t h e i r a b i l i t y t o u n d e r s t a n d t h e i m p o r t a n c e o fc o n t r o l s , r e la t e e x p e r i m e n t a l r e s u l t s t o t h e b i g g e r p i c t u r e i nt he i r r e spec t i ve f i e l ds , and s t a t i s t i ca l l y ana l yze da t a . T ha ti n t e r n s a p p a r e n t l y o v e r e s t i m a t e d t h e l e v e l s o f t h r e e o f t h e i rs c i en t i f i c sk i l l s i s no t pa r t i cu l a r l y su r pr i s i ng . F or exampl e ,A t w a t e r a n d Y a m m a r i n o ( 1 9 9 7 ) h a v e n o t e d t h a t m o s tr e s e a r c h t h a t h a s e x a m i n e d s e l f - o t h e r r a t i n g s o f t r a i t s ,a b i li ti e s , p e r f o r m a n c e , o r l e a d e r s h i p h a s f o u n d s e l f -r a t in g s t ob e t y p i c a l l y h i g h e r t h a n r a t i n g s p r o v i d e d b y o b s e r v e r s .M o r e o v e r , r e s e a r c h e r s s u c h a s P a u l u s ( a s c it e d i n A t w a t e r& Y a m m a r i n o , 1 9 9 7 ) h a v e a r g u e d t h a t s u c h a s e l f - e n h a n c e -m e n t b i a s m a y b e p s y c h o l o g i c a l l y h e a l t h y i n t h a t i t r e f le c t sp o s i t i v e s e l f - e v a lu a t i o n a n d r e s u l t s i n f e w e r n e g a t i v e t h o u g h t sa n d h i g h e r e x p e c t a n c i e s f o r s u c c e s s in n e w e n d e a v o r s .W h a t i s m o r e s u r p r i s i n g i n t h e p r e s e n t s t u d y i s t h e s t r ik i n ga g r e e m e n t b e t w e e n m e n t o r s ' a n d i n t e r n s ' r a t in g s o f 1 1 o f t h ei n t e r n s ' s k i ll l e v e ls a t th e e n d o f th e U R E . A l t h o u g h b o t h s e l fa n d o t h e r r a t i n g s c a n b e i n f l u e n c e d b y a n u m b e r o fi n d i v i d u a l a n d c o n t e x t u a l f a c t o rs , r e s e a r c h e r s h a v e g e n e r a l l ya c k n o w l e d g e d t h a t t h e d e g r e e t o w h i c h o n e ' s s e l f - r a t i n g sm a t c h a c r i t e r i o n m e a s u r e r e p r e s e n t s a n i n d i c a t o r o f s e l f -r a t in g a c c u r a c y ( A t w a t e r & Y a m m a r i n o , 1 9 9 7 ). O n t h e b a s is

o f t h e i n t e r n - m e n t o r l e v e l o f a g r e e m e n t i n t h i s s t u d y , i ta p p e a r s t h a t t h e u n d e r g r a d u a t e i n t e r n s h a d a g o o d s e n s e o ft h e i r a b s o l u t e l e v e l o f p e r f o r m a n c e . I n t e re s t in g l y , t h i s f i n d -i n g i s c o n s i s t e n t w i th f i n d i n g s f r o m F a l c h i k o v a n d B o u d ' s( 1 9 8 9 ) m e t a - a n a l y s i s o f 4 8 s t u d ie s t h a t e x a m i n e d t h e d e g r e eo f c o r r e s p o n d e n c e b e t w e e n s t u d e n t a n d te a c h e r r a ti n g s o fs t u d e n t p e r f o r m a n c e . T h e s e r e s e a r c h e r s f o u n d t h a t s t u d i e sw i t h i n s c i e n c e p r o d u c e d m o r e a c c u r a t e s e l f - a s s e s s m e n t st h a n t h o s e f r o m a n y o t h e r a c a d e m i c d i s c ip l i n e.G e n d e r D i f f e r e n c e s i n I n t e r n s S e l f - R a t i n g s

T a b l e 3 p r e s e n t s m e a n s a n d s t a n d a r d d e v i a t i o n s f o r m a l ea n d f e m a l e i n t e r n s ' s e l f - r a ti n g s o f t h e i r a b i l it y t o p e r f o r m t h er e s e a r c h s k i l ls a t th e b e g i n n i n g o f t h e U R E . R e s u l t s o fi n d e p e n d e n t - s a m p l e s t t e s t s r e v e a l t h a t m a l e a n d f e m a l ei n t e r ns d i d no t d i f f e r s i gn i f i can t l y i n t he i r r a t i ngs o f sk i l ll e v e l s a t t h e b e g i n n i n g o f t h e U R E . T a b l e 4 p r e s e n ts t h em e a n s a n d s t a n d a r d d e v i a t i o n s f o r i n t e r n s ' ra t i n g s a t t h e e n do f t h e U R E a s a f u n c t io n o f g e n d e r. A s i n d i c a t e d b y t h er e s u l ts i n t h e f i r st c o l u m n o f t t e st s , b y t h e e n d o f t h e U R E ,m a l e i n t e r n s r a t e d t h e m s e l v e s s i g n i f i c a n t l y h i g h e r t h a n d i df e m a l e i n t e r n s w i t h r e s p e c t t o t h e i r a b i l i t y t o u n d e r s t a n dc o n t e m p o r a r y c o n c e p t s i n t h e i r f ie l d. T h e r e w e r e n o s i g n if i -c a n t g e n d e r d i f f e r e n c e s o n a n y o f t h e o t h e r r e s e a r c h s k i l ls .A n a d d i t io n a l a n a l y s i s i n v o l v e d e x a m i n a t i o n o f p r e - top o s t - U R E c h a n g e s i n r a ti n g s o f s k i l l l e v e l s a s a f u n c t io n o fg e n d e r . R e s u l t s o f i n d e p e n d e n t - s a m p l e t t e s t s r e v e a l t h a tm e n p e r c e i v e d s i g n i f i c a n t l y m o r e o f a n i n c r e a s e t h a n d i dw o m e n i n th e i r a b il i ty t o u n d e r s t a n d c o n t e m p o r a r y c o n c e p t si n t h e i r f i e l d ( m e a n d i f f e r e n c e s o f - 1 . 0 4 a n d - 0 . 3 4 ,

T a b l e 3Descr ip t i ve S ta t i s t ic s and t T es ts fo r D i f f e rences in Ma le an d Fem ale I n te rns ' Se l f - Rat ingso f T he i r Research Sk i l l s a t the Beg inn ing o f the U ndergraduate Research Exper iencePerceived level of skill

W omen M enI t em M SD M SD t(55)

Understand contemporary concepts in you r field 3.24 0.64Ma ke use of the p rimary scientific research l i tera-ture in you r field (e.g., journa l art icles) 3.13 0.99Identify a specific question for investigation basedon the research in you r field 2.84 0.87Formulate a research hypothesis base d on a spe-cific question 2.95 0.88Design a n experiment or theoretical test of thehypo thesis 2.65 1.04Understand the importance of con trols inresearch 4.27 0.55Ob serve and collect data 4.12 0.82Statistically analyze data 3.02 1.15Interpret data by relating results to the originalhypothesis 2.97 0.85Reformulate y our original research hypothesis (asapprop riate) 2.96 0.77Relate results to the b igg er picture in you r field 2.90 0.95Orally comm unicate the results of research projects 3.07 0.90Write a research pap er for publication 2.21 0.77

3.13 0.58 0.633.07 0.75 0.272 .86 0 .80 - 0 .092.71 0.73 1.082.62 0.84 0.144.21 0.69 0.363.88 0.74 1.153.29 0.89 -0.973 .04 0 .55 - 0 .362.95 0.62 0.092 .93 0 .75 - 0 .143.06 0.69 0.032.39 0.78 -0.83

Note. Each i tem completes the question To what extent do you feel you canrated on a 5-point scale ranging fro m 1 (not at all) to 5 (a great deal). ? I t e m s w e r e


8/11

198 KARDASHT a b l e 4D e s c r i p t i v e S t a ti s ti c s a n d t T e st s f o r I n t e r n s S e l f -R a t i n g s a n d M e n t o r s R a t i n g s o f I n t e r n s R e s e a r c h S k i ll s a t th e E n d o f th eU n d e r g r a d u a t e R e s e a r c h E x p e r i e n c e a s a F u n c t i o n o f I n t e r n G e n d e r

Intern s' self-ratings M entors ' ra tingsW o m e n M e n W o m e n M e n

I t e m M S D M S D t 5 5 ) a M S D M S D t(55) bUnde r s t a nd c on t e mpor a r y c onc e p t s i n you rf ield 3 .57 0 .79 4 .17 0 .76 -2 .86 *M ake u se of the primary scientific researchliterature in yo ur field (e.g. , journ alarticles) 3.84 0.87 3.46 1.02 1.52Iden tify a specific que stion fo r investiga-t ion based on the research in you r f ield 3 .37 0 .86 3 .2 1 0 .93 0 .69Form ula te a research hypothes is based ona specific ques tion 3.34 0.81 3. 71 0.75 - 1.73Design an exp er iment or theore t ica l tes t ofthe hypothes is 3 .22 0 .99 3 .34 0 .87 -0 .4 9Unde r s t and the i mpor t a nc e o f c on t r o l sin research 4 .47 0 .79 4 .56 0 .65 -0 .4 7Observe and col lec t da ta 4 .45 0 .79 4 .48 0 .65 -0 .1 2Statist ically analy ze data 3.56 1.06 3.99 0.78 - 1.68Interpre t data by relating results to theor ig inal hypothes is 3 .88 0 .82 4 .00 0 .72 -0 .5 9Reform ula te your or ig ina l researchhypo thesis (as appropriate) 3.64 0.96 3.55 0.71 0.37Rela te resul ts to the b igg er p ic ture inyo ur field 3.76 0.97 3.75 0.99 0.03Oral ly com mun icate the resul ts of researchprojects 3.91 0.80 3.79 0.93 0.51Write a research pape r for publication 2.96 0.81 2.95 1.20 0.04Think independent ly 4 .03 0 .92 4 .29 0 .62 - 1 .20

3.76 0.87 3.71 0.69 0.23

3 . 21 0 . 96 3 .33 0 . 92 - 0 . 483 .22 0 . 99 3 .43 1 . 01 - 0 . 8 03.25 0.94 3.66 0.92 - 1.663.44 1 .09 3 .54 0 .98 -0 .374.03 0.77 4.00 0.72 0.154.45 0.75 4.38 0.58 0.433 .1 4 1 . 2 1 3 .5 4 1 . 0 5 - 1 . 0 0 ( d f = 33)3.93 0.79 3.71 0.86 1.053.31 1 .07 3 .55 0 .86 -0 .84 ( d f = 49)3.09 1 .10 3 .33 0 .92 -0 .884.00 0 .94 3 .87 0 .74 0 .542 .8 2 1 . 1 8 2 .8 7 0 . 9 9 - 0 . 1 3 ( d f = 35)3.78 0 .96 3 .79 1 .06 -0 .04

Note. Ea c h i t e m c ompl e te s the que s ti on To wha t e x t en t dor a ng i ng fr om 1 (not at all) to 5 (a great deal).at test for differences in m ale and fem ale intern s' self-ratings.*p < .01, two-tailed.you fee l you (your in tern) can ? I tems were ra ted on a 5-point sca lebt test for differences in men tors ' ratings o f male and f em ale interns.

r e s p e c t i v e l y ) , t ( 5 5 ) = 3 . 0 1 , p < . 0 1 . M o r e o v e r , t h e r e w a s am a r g i n a l l y s i g n i f i c a n t g e n d e r d i f f e r e n c e i n p r e - t o p o s t -r a ti n g s o f t h e a b i l it y t o f o r m r e s e a r c h h y p o t h e s e s . M e nt e n d e d t o p e r c e i v e m o r e o f a n i n c r e a s e i n th i s s k i ll t h a n d i dw o m e n ( m e a n d if f er e n ce s o f - 1 . 0 0 a n d - 0 . 3 9 , r e sp e c -t i ve l y ) , t ( 55 ) = 2 . 53 , p = . 014 . A l t ho ug h t h i s l a t t e r f i nd i ngm u s t b e i n t e r p r e t e d w i t h c a u t i o n , i t d o e s p a r a l l e l a f i n d i n gt h a t e m e r g e d i n a p r e v i o u s s t u d y ( K a r d a s h , 1 9 9 9 ) . I n th a ts t u d y , 1 1 0 i n t e r n s w e r e a s k e d t o l i st th e m a j o r d r a w b a c k s o rb a r r i e r s a s s o c i a t e d w i t h p u r s u i n g a c a r e e r a s a r e s e a r c hs c i e n ti s t. A t t h e b e g i n n i n g o f t h e U R E , s i g n i f i c a n t l y m o r ef e m a l e t h a n m a l e i n te r n s m e n t i o n e d a b i l i ty t o g e n e r a t er e s e a rc h h y p o t h e s e s . A p p a r e n t l y , f e m a l e i n te r n s, m o r e t h a nm a l e i n t e r n s , s e e m t o h a r b o r d o u b t s a b o u t t h e i r a b i l i t y t og e n e r a t e r e s e a r c h h y p o t h e s e s . T h i s i s a p o i n t t h a t f a c u l t ym e n t o r s s h o u l d b e e s p e c i a l l y s e n s i t i v e t o , p e r h a p s b ye n s u r i n g t h a t f e m a l e i n t e r n s h a v e a m p l e o p p o r t u n i t y t op r a c t i c e t h i s p a r t i c u l a r s k i l l . F e m a l e i n t e r n s ' t e n d e n c y t ou n d e r e s t i m a t e t h e i r o w n a b i li t y t o g e n e r a te r e s e a r c h h y p o t h -e s e s ( i f n o t c o r r e c t e d b y f e e d b a c k f r o m m e n t o r s a b o u t t h e irl e v e l o f p e r f o r m a n c e ) m a y u l t i m a t e ly u n d e r m i n e t h e irc o n f i d e n c e i n t h e i r a b i l i t y t o c o n d u c t s c i e n t if i c r e s e a r c h .

O v e r a l l , h o w e v e r , m a l e a n d f e m a l e i n t e r n s ra t e d t h e i r sk i l ll e v e ls s i m i l a rl y a t b o t h t h e b e g i n n i n g a n d e n d o f t h e U R E .T h i s f i n d i n g i s c o n s i s t e n t w i t h s o m e o f th e r e s e a r c h o n

g e n d e r d i f f e r e n c e s i n s e l f - r a t i n g s i n e d u c a t i o n a l a c h i e v e -m e n t s e t t in g s . I n g e n e r a l , th e s e s t u d i e s h a v e f o u n d t h a t w h e nr a t i n g s a r e p r o v i d e d c o n f i d e n t i a l l y ( a s t h e y w e r e i n t h i ss t u d y) , w o m e n d o n o t r a te t h e m s e l v e s l o w e r th a n m e n( A t w a t e r & Y a m m a r i n o , 1 9 9 7 ) . M o r e o v e r , t h e l a c k o f g e n d e rd i f f e r e n c e s i s e n c o u r a g i n g i n te r m s o f i t s i m p l i c a t i o n s f o rp e r s is t e n c e r a te s . S e y m o u r a n d H e w i t t ( 1 9 9 7 ) h a v e n o t e dt h a t g e n d e r - b a s e d d i f f e re n c e s i n p e r s is t e n c e r a t e s f o r S M E Tm a j o r s t e n d t o d i s a p p e a r i n c a s e s w h e r e m a l e a n d f e m a l eS M E T m a j o r s e x h i b it s i m i l ar l ev e l s o f a c a d e m i c p e r f o r m a n c e .

M e n t o r s R a t i n g s o f M a l e a n d F e m a l e I n te r n sT a b l e 4 a l s o p r e s e n t s m e a n s a n d s t a n d a r d d e v i a t i o n s f o rm e n t o r s ' r a t i n g s o f t h e ir r e s p e c t i v e i n t e r n s ' s k i ll s a s a

f u n c t i o n o f i n te r n g e n d e r . A s d i s p l a y e d i n t h e s e c o n d c o l u m no f t t e s ts , m e n t o r s r a t e d m a l e a n d f e m a l e i n t e r n s e s s e n t i a l lyt h e s a m e o n a l l 1 4 s k il l s a t t h e e n d o f t h e U R E . T h i s l a c k o fg e n d e r - b a s e d d i f f e r e n c e s i n m e n t o r s ' r a t in g s o f i n t e rn s k i l ll e v e l s i s c o n s i s t e n t w i t h S e y m o u r a n d H e w i t t ' s ( 1 9 9 7 )f i n d i n g t h a t f e m a l e S M E T m a j o r s g e n e r a l l y r e p o r t e d n om a r k e d d i f f e r e n c e s i n t h e w a y s t h a t f e m a l e a n d m a l es t u d en t s w e r e t r e at e d b y a m a j o r i ty o f S M E T f a c u lt y .

T o d e t e r m i n e w h e t h e r m a l e a n d f e m a l e i n t e rn s d i f f e r e d int h e i r a b i l it y t o a c c u r a t e l y a s s e s s t h e i r o w n s k i l l l e v e l s a t th e


9/11

EVALUATION OF URE 19 9e n d o f t h e U R E , a c c u r a c y r a ti n g s w e r e c a l c u l a t e d b ysub t r ac t i ng e ach i n t e r n ' s s e l f - r a t i ng f o r the 14 sk i l ls a t thee n d o f t h e U R E f r o m h i s o r h e r r e s p e c ti v e m e n t o r ' s r a t in g o ft h o s e s a m e s k i l l s . M e a n s a n d s t a n d a r d d e v i a t i o n s f o r t h e s ea c c u r a c y r a t i n g s , p r e s e n t e d s e p a r a t e l y f o r e a c h g e n d e r ,a p p e a r in T a b l e 5 . R e s u l t s o f d e p e n d e n t - s a m p l e t t e st s r e v e a ls t r i k i n g a g r e e m e n t b e t w e e n m e n t o r s ' a n d i n t e r n s ' r a t i n g s ,r ega r d l e s s o f i n t e r n gender . I n f ac t , t he on l y s i gn i f i can td i f f e r e n c e t h a t e m e r g e d w a s t h a t f e m a l e i n t e r n s r a t e dt h e m s e l v e s s i g n i f i c a n t l y h i g h e r t h a n d i d t h e i r r e s p e c t i v em e n t o r s w i t h r e s p e c t t o t h e i r a b i l i t y t o m a k e u s e o f t h epr i m ar y sc i en t i fi c l i t e r a t u r e i n t he i r f i e ld .S u m m a r y , C o n c l u s io n s , a n d L i m i t a t i o n s

R i c h m o n d ( 1 9 9 8 ) h a s a r g u e d t h a t a ll s tu d e n t s a r e c a p a b l eo f d e v e l o p i n g s k i ll s a s s o c i a t e d w i t h s c i e n t if i c u n d e r s t a n d i n gand p r ob l em so l v i ng , and t ha t pa r t i c i pa t i on i n a s c i en t i f i cc o m m u n i t y h e l p s s t u d e n t s t o d e v e l o p a v i e w o f t h e c u l t u r ea n d p r a c t i c e o f s c i e n c e t h a t e n h a n c e s t h a t u n d e r s t a n d i n g .H o w e v e r , s h e a l s o a s k e d , D o w e , i n a n y p r e s e n t - d a ye d u c a t i o n a l i n s t i t u t i o n s , e v e n h a v e m o d e l s f o r s u c h p r o -c e s s e s - - p r o g r a m s , e n v i ro n m e n t s , a n d st r a te g i e s w h i c h e n c u l -t u r a t e i n d i v i d u a l s i n t o t h e e n t e r p r i s e , p r o v i d i n g t h e m w i t ht h e t e c h n i c a l , s o c i a l , a n d c o g n i t i v e t o o l s t o b e c o m e a c c o m -p l i s h e d b e g i n n e r s ? ( p . 5 8 4 ) . S im i l a r ly , o th e r s c i e n c e r e -s e a r c h e r s h a v e c a l l e d f o r m o d e l s o f l e a r n i n g i n r e s e a r c hl a b o r a t o r i e s t h a t p r e s e n t a c l e a r e r p i c t u r e o f w h a t e x a c t l yt r a n s p i r e s i n t h e s e l a b o r a t o r i e s a n d w h a t a n d h o w s t u d e n t sc a n l e a r n in t h e m ( S h y m a n s k y & K y l e , 1 9 9 2 ).

R e s u l t s o f t h e p r e s e n t s t u d y b e g i n t o a d d r e s s t h i s n e e d f o rm o d e l s o f l e a r n i n g i n t h e r e s e a r c h l a b o r a t o r y b y d e s c r i b i n g ,f r o m t h e p e r s p e c t i v e o f b o t h u n d e r g r a d u a t e i n t e rn s a n d t h e i rf a c u l t y m e n t o r s , t h e e x t e n t t o w h i c h U R E s e n h a n c e i n t e r n s '

a b i l it i es t o e n g a g e i n t h e r e a l w o r k o f s c i en t is t s: l e a r n i n gto concep tua l ize r ea l s c ien t i f i c p r ob lem s , des ign expe r i -m ents , p e r f o r m s k i l l s n e c e s s a r y t o a n a l y z e a n d c o l l e c t d a t a ,a n d d r a w c o n c l u s i o n s , w h i l e l e a r n i n g t o s e e b e y o n d r e s u lt st o th e l a r g e r c o n t e x t in w h i c h t h e p r o b l e m r e s t s ( R i c h m o n d ,1998 , p . 584) . T h e r e su l t s o f t h i s s t udy a l so add t o t hee m e r g i n g l i t e r a t u r e d o c u m e n t i n g t h e e f f e c t s o f a u t h e n t i cl e a r n in g e x p e r i e n c e s o n i n d i v id u a l s l e a r n in g a n d a c h i e v e -m e n t . O v e r a l l , t w o s o u r c e s o f e v i d e n c e s u p p o r t t h e c o n t e n -t i o n th a t U R E s d o h a v e a p o s i t i v e i m p a c t o n d e v e l o p m e n t o fu n d e r g r a d u a t e s r e s e a r c h s k i l l s. F i r s t , t h e i n t er n s t h e m s e l v e sp e r c e i v e d a s i g n i f i c a n t i n c r e a s e i n t h e i r a b i l i t y t o p e r f o r m t h er e s ea r c h s k i l ls f r o m t h e b e g i n n i n g t o e n d o f t h e U R E .S e c o n d , a n y w e a k n e s s i n h e r e n t i n t h e s e l f -r e p o r t n a t u r e o ft h e s e d a t a i s m i t i g a t e d s o m e w h a t b y t h e f i n d i n g th a t i n t e r n ss e l f - a s s e s s m e n t s a t t h e e n d o f t h e U R E d i d n o t d i ff er , f o r t h em o s t p a rt , f r o m t h e i r f a c u l t y m e n t o r s a s s e s s m e n t s o fin terns sk i l l l eve l s .

T w o l i m i t a t i o n s o f t h e p r e s e n t s t u d y a r e t h e l a c k o f ac o n t r o l g r o u p a n d t h e s e l f - r e p o r t n a t u r e o f t h e i n s t r u m e n tused to a s ses s l eve l o f r e sea r ch sk i ll s . Use o f a con t r o l g r oupw o u l d h a v e s t r e n g t h e n e d th e s t u d y f r o m a m e t h o d o l o g i c a lp e r s p e c t i v e , a n d f u t u r e s t u d i e s m i g h t i n c l u d e a m a t c h e dg r o u p o f s c ie n c e m a j o r s w h o h a d t a k e n t h e s a m e c l a s se s b u td i d n o t p a r t i c ip a t e i n t h e U R E . A s p o i n t e d o u t b y o n e o f t h er e v i e w e r s o f th i s s tu d y , s u c h a c o n t r o l g r o u p w o u l d a l l o w f o ri n v e s t i g a t io n o f s p e c i fi c e f f e c t s o f t h e U R E o v e r a n d a b o v et h o s e a c q u i r e d d u r i n g t h e c o u r s e o f u n d e r g r a d u a t e s t u d i e s.R e g a r d i n g t h e s e c o n d l i m i ta t i o n , th e u s e o f p r e - a n dp o s t - U R E s e l f - r a t i n g s r a i s e s t h e q u e s t i o n o f a p o t e n t i a lr e s p o n s e s h i ft b i a s i n t h e d a ta . A s p o i n t e d o u t b y a r e v i e w e r ,one way t o addr es s t h i s i s sue i n f u t u r e s t ud i e s i s t o e l i mi na t et he p r e t e s t . I ns t ead , i n t e r ns w oul d r a t e t he i r sk i l l l eve l s a t t he

T a b l e 5Descr ip t i ve S ta t i s ti c s and Pa ired t T es ts fo r Accura cy Rat in gs as a Fun c t ion o f I n te rn Gen der

I t em M

Accuracy ratingsFem ale interns Male internsSD t(32) a M S D t(23) b

Understand contemporary concepts in you r field 0.19Ma ke use of the prim ary scientific research l i terature in yo ur field(e.g., journal art icles) -0 .6 3Identify a specific question for investigation based on the researchin your f ie ld -0 .15Formulate a research hypothesis base d on a specific question -0 .0 9Design an experiment or theoretical test of the hypothesis 0.22Understand the importance of cont rols in research -0 .4 4Ob serve and collect data 0.00Statist ically analyze data -0 .6 1Interpret data by relating results to the original hypo thesis 0.06Reformulate you r original research hypothesis (as appropriate) -0 .3 1Relate resul ts to the bigger pic ture in your f ie ld -0 .6 7Orally com mu nicate the results of research projects 0.09Write a research pap er for publication -0 .0 8Think independently -0 .25

1 .1 5 0 .9 4 - 0 . 4 5 1 . 0 6 - 2 . 1 11 . 2 3 - 2 . 9 3 * - 0 . 1 2 1 . 4 8 - 0 . 4 11 .38 - 0 .6 4 0 .22 1 .44 0 .761 . 1 8 - 0 . 4 7 - 0 . 0 5 1 . 1 6 - 0 . 2 01.57 0.80 0.20 1.4 6 0.670 .9 8 - 2 . 5 7 - 0 . 5 6 1 . 1 0 - 2 . 5 21 .09 0 .00 - 0 .1 0 0 .86 - 0 .5 91 . 41 - 2 . 0 4 ( d f = 2 1) - 0 . 7 7 1 . 4 8 - 1 . 8 71 .2 2 0 .2 8 - 0 . 2 9 1 . 1 2 - 1 . 2 91 .2 3 - 1 . 3 6 ( d f = 2 8) - 0 . 0 5 1 . 2 8 - 0 . 2 01 . 4 5 - 2 . 6 4 - 0 . 4 2 1 . 5 0 - 1 . 3 61.18 0.44 0.08 1.2 8 0.321 .3 1 - 0 . 3 1 ( d f = 2 1) - 0 . 2 6 1 . 2 7 - 0 . 7 91 . 1 0 - 1 . 3 1 - 0 . 5 0 1 . 2 5 - 1 . 9 6

d f = 12)d f = 21)d f = 14)

Note . Each i tem completes the question To what extent do you feel you (your intern) can ? Item s were rated on a 5-point scaleranging from 1 not at all) to 5 a great deal).at test for differences in fem ale interns' and m atched m entors ' rat ings, bt test for differences in male interns' and matched m ento rs ' rat ings.*p < .01, two-tailed.


10/11

2 0 0 K A R D A S He n d o f t h e U R E a n d c o m p a r e e a c h s k i l l r a t in g t o h o w i t w a sp r i o r t o p a r t i c i p a t i o n i n t h e U R E . T h a t p a r t i c u l a r s t r a t e g yw a s d e l i b e r a t e l y n o t u s e d i n t h e p r e s e n t s t u d y b e c a u s ei n t e r n s w e r e a w a r e t h a t th e s t u d y w a s p a r t o f a n e v a l u a t i o ne f f o r t. T h u s , t h e r e w a s a p o s s i b i l i t y t h a t s tu d e n t s m i g h t s k e wt h e i r ra t i n g s i n s u c h a m a n n e r a s t o p r e s e n t t h e m o s tf a v o r a b l e p i c t u r e p o s s i b l e t o t h e a g e n c i e s t h a t f u n d e d t h eU R E s i n a n a t t e m p t t o e n s u r e f u t u r e f u n d i n g .

I n s u m m a r y , t h e 1 4 - i t e m i n s t r u m e n t d e s c r i b e d i n t h i ss t u d y p r e s e n t s a f i r st s te p t o w a r d p r o v i d i n g a n o b j e c t i v ea s s e s s m e n t o f t h e e f fi c a c y o f U R E s a s a l e a r n in g e x p e r i e n c e .T h e s c a l e y i e l d e d h i g h i n t e r n a l c o n s i s t e n c y e s t i m a t e s w h e nu s e d w i t h b o t h i n t e r n s a n d f a c u l t y m e n t o r s , a n d t h e f i n d i n go f s i g n i f i c a n t p r e t e s t a n d p o s t t e s t d i f f e r e n c e s i n i n t e r n s 's e l f - r a t i n g s i n d i c a t e s t h a t t h e i n s t r u m e n t h a s p o t e n t i a l c o n -s t r u c t v a l i d i t y ( C o h e n , S w e r d i i k , P h i l l ip s , 1 9 96 ) . T h e s i z eo f t h e s a m p l e i n t h is s t u d y p r e c l u d e d a f a c t o r a n a l y s is o f t h ei n s t r u m e n t , a l t h o u g h s u c h a n a n a l y s i s s h o u l d b e u n d e r t a k e ni n t h e f u t u r e t o p r o v i d e f u r t h e r i n f o r m a t i o n r e g a r d i n gt h e i n s t r u m e n t ' s p s y c h o m e t r i c p r o p e r t i e s a n d u n d e r l y i n gc o n s t r u c t s .

G u b a a n d L i n c o l n ( a s c i t e d i n D o r i H e r s c o v i t z , 1 9 9 9)h a v e d e f i n e d e v a l u a t i o n a s t h e i n t e g r a t i o n o f b o t h d e s c r i p -t i o n a n d j u d g m e n t , i n w h i c h t h e d e s c r i p t io n e m p h a s i z e s t h eo b j e c t iv e p a r t o f t h e a s s e s sm e n t , a n d t h e j u d g m e n t p a r td w e l l s o n i t s s u b j e c t i v e a s p e c t s . T h e 1 4 - i t e m i n s t r u m e n tp r o v i d e s a m e a n s f o r p o t e n t i a l ly e n h a n c i n g t h e o b j e c t i v i t y o ft h e d e s c r i p t i v e p a r t o f t h e e v a l u a t i o n . I n a n y c a s e , d a t a s u c ha s t h o s e p r o v i d e d i n t h i s s t u d y c a n s e r v e a s b o t h as u m m a t i v e a s s e s s m e n t o f i n t e r n s ' a c q u i s it i o n o f s p e c i fi cr e s e a r c h s k i l l s d u r i n g t h e t i m e t h a t i n t e r n s a r e a c t u a l l ye n g a g e d i n r e s e a r c h e x p e r i e n c e s a s w e l l a s f o r m a t i v ef e e d b a c k t o in t e r n s r e g a r d i n g w h i c h r e s e a r c h s k i ll s th e yh a v e a c q u i r e d , a n d w h i c h r e q u i r e f u r t h e r i m p r o v e m e n t .

R e f e r e n c e sA h l m , K . ( 1 9 9 7) . A r e s e a r c h e r ' s g u i d e t o k e y a s s e s s m e n t p r i n c i p l e sand techn iques . Cou ncil on Undergraduate Rese arch Quarterly ,18, 7 0 - 7 2 , 9 0 .A t w a t e r , L . E . , Y a m m a r i n o , E J . ( 1 99 7 ). S e l f - o t h e r r a t i n ga g r e e m e n t : A r e v i e w a n d m o d e l . Research in Personne l andHum an Resources Managem ent , 15 , 1 2 1 - 1 7 4 .B e t z , N . E . ( 1 9 94 ) . C a r e e r c o u n s e l i n g f o r w o m e n i n t h e s c i e n c e sa n d e n g i n e e r i n g . I n W . B . W a l s h S . H . O s i p o w ( E d s . ) , C a r e e rc o u n s e li n g f o r w o m e n ( p p . 2 3 7 - 2 6 2 ) . H i l ls d a l e , N J : E r l b a u m .Bloc kus , L . , Kard ash , C. M. , Bla i r , M. , Wa l lace , M . (1997).U n d e r g r a d u a t e i n t e r n s h i p p r o g r a m e v a l u a t i o n : A c o m p r e h e n s i v ea p p r o a c h a t a r e s e a r c h u n i v e r s i t y . Council on UndergraduateRese arch Quarterly, 18, 60--63, 91.Brow n , J . S . , Co l l in s , A. , Dug u id , P . (1989). S i tua ted cogn i t iona n d t h e c u l t u r e o f l e a r n i n g . Educ ational Researcher, 18, 32--42.Brush , S . G. (1991) . Women in sc ience and eng inee r ing . A m e r i c a nScien tist, 79, 40 a. A 19.Co hen , R. J . , Sw erd l ik , M . E. , Ph i l l ip s , S . M . (1996). Psycho log i -ca l t e s t ing and assessment : An in t roduct ion to t e s t s and m easure -m e n t ( 3 r d e d .) . M o u n t a i n V i e w , C A : M a y f i e l d .C o n s o r t i u m f o r P o l i c y R e s e a r c h i n E d u c a t i o n . ( 1 9 9 5 , M a y ) .Reforming sc ience , mathemat ics , and techno logy educa t ion:NSF s s ta te sy s temic in i t ia t ives . ( C P R E P o l i cy B r i e f N o . R P - 1 5 -

M a y 1 9 9 5 ) . N e w B r u n s w i c k , N J : R u t g e r s U n i v e r s i t y , C a r r i a g eH o u s e a t t h e E a g l e t o n I n s t i tu t e o f P o l i t i c s.D a v i s , D . S . , G l a z i e r , D . (1 9 97 ) . H o w d o w e e v a l u a t e u n d e r g r a d u -a t e r e s e a r c h ? Cou ncil on Undergraduate Rese arch Quarterly ,18, 7 3 - 7 6 .D o r i , Y . J . , H e r s c o v i t z , O . ( 1 9 9 9) . Q u e s t i o n - p o s i n g c a p a b i li t y a sa n a l t e r n a t iv e e v a l u a t i o n m e t h o d : A n a l y s i s o f a n e n v i r o n m e n t a lcase s tudy . Journa l o f Research in Sc ience Teaching, 36 ,411--430.Ecc les , J . , Ka cza la , C . M. , M eece , J . L . (1982). Soc ia l iza t ion o fa c h i e v e m e n t a t t it u d e s a n d b e l i e f s: C l a s s r o o m i n f lu e n c e s . C h i MDeve lopment , 53 , 3 2 2 - 3 3 9 .Fa lch ikov , N. , Bou d , D. (1989). S tuden t se l f -a sse ssm en t inh i g h e r e d u c a t i o n : A m e t a - a n a l y s i s . R e v i e w o f E d u c a ti o n a l R e -search, 59, 395--430.F a r m e r , H . S . (1 9 9 7) . W h y w o m e n d o n ' t p e r s i s t i n t h e i r h i g h s c h o o lc a r e e r a s p i r a t i o n s f o r c a r e e r s i n s c i e n c e o r t e c h n o l o g y . I n H . S .F a r m e r a n d a s s o c i a t e s ( E d s . ) , D i v e r s i t y a n d w o m e n s c a r e e rdeve lopment : From ado lescence to adu l thood ( p p . 6 2 - 8 0 ) .T h o u s a n d O a k s , C A : S a g e .Fe r re i ra , M. M. (1998 , Apr i l ) . Gender i s sues re la ted to s tuden ta t t r i tion f rom two gradua te sc ience depar tmen ts . P a p e r p r e -s e n t e d a t t h e a n n u a l m e e t i n g o f t h e A m e r i c a n E d u c a t i o n a lR e s e a r c h A s s o c i a t i o n , S a n D i e g o , C A .Ga gne , E . D. , Yek ov ich , C. W. , Yek ov ich , E R. (1993). Thecogn i t i ve psycho logy o f schoo l l earn ing . N e w Y o r k : H a r p e r -Co l l in s .H a k i m , T . (1 9 9 8) . S o f t a s s e s s m e n t o f u n d e r g r a d u a te r e s e a rc h :R e a c t i o n s a n d s t u d e n t p e r s p e c t iv e s . Council on UndergraduateRese arch Quarterly, 18, 1 8 9 - 1 9 2 .Ha l l , R . M . , Sand ie r , B . (1982). The c lassroom c l imate : A ch i l l yo n e f o r w o m e n ? P r o j e c t o n t h e S t a t u s o f W o m e n . W a s h i n g t o n ,D C : A s s o c i a t i o n o f A m e r i c a n C o l l e g e s a n d U n i v e r s it i e s .Kah le , J . B . (1990) . Rea l s tuden ts take chemis t ry and phys ic s :Ge nde r i s sues . In K. Tob in , J . B . Kah le , B. J . F rase r (Eds . ) ,Windows in to sc ience c lassrooms: Prob lems assoc ia ted wi thh igher - leve l cogn i t i ve l earn ing ( p p . 9 2 - 1 3 4 ) . N e w Y o r k: F a l m e r .Kah le , J . B . , Renn ie , L . J . (1993). Am el io ra t ing gend erd i f fe rences in a t t i tudes abou t sc ience : A c ross -na t iona l s tudy .Journa l o f Sc ience Educa t ion and Techno logy, 23 , 3 2 1 - 3 3 3 .K a r d a s h , C . M . ( 1 9 9 9 , A p r i l ) . A research sc ience career : Perspec-t i ves o f sc ience acu l ty and the i r undergradua te research interns.P a p e r p r e s e n t e d a t t h e A n n u a l M e e t i n g o f t h e A m e r i c a n E d u c a -t i o n a l R e s e a r c h A s s o c i a t i o n , M o n t r e a l , C a n a d a .Kob a l la , T . , Bu t t s , D. , Ri ley , J . (1995 , Apr i l ) . An eva lua t ionmode l w i th par t i c ipan t co ns t ruc ted componen ts . P a p e r p r e s e n t e da t t h e a n n u a l m e e t i n g o f t h e N a t i o n a l A s s o c i a t i o n f o r R e s e a r c h i nS c i e n c e T e a c h i n g , S a n F r a n c i s c o , C A .K r e m e r , J . E , B r i n g l e , R . G . ( 1 99 0 ). T h e e ff e c t s o f a n i n t e n s i v er e s e a r c h e x p e r i e n c e o n t h e c a r e e r s o f t a l e n t e d u n d e r g r a d u a t e s .Journa l o f Research and D eve lopm ent in Educa tion, 24 , 1-5 .Lave , J . (1995 , Apr i l ) . What s the s i tua t ion o f l earn ing (a f t e r twod e c a d e s o f p r a c t i c e )? P a p e r p r e s e n t e d a t t h e a n n u a l m e e t i n go f t h e A m e r i c a n E d u c a t i o n a l R e s e a r c h A s s o c i a t i o n , S a nF r a n c i s c o , C A .L e a c h , L . S . ( 1 99 5 ). S e x u a l h a r a s s m e n t i n c h e m i s t r y c l a s s r o o m s :T h r e e s t u d e n t s ' e x p e r i e n c e s . Schoo l Sc ience and Mathemat ics ,95, 3 2 0 - 3 2 6 .M a n d u c a , C . ( 1 9 9 7 ). B r o a d l y d e f i n e d g o a l s f o r u n d e r g r a d u a t er e s e a r c h p r o j e c t s : A b a s i s f o r p r o g r a m e v a l u a t i o n . Counc i l onUndergraduate Rese arch Quarterly, 18, 64-69 .Nat iona l Sc ience Founda t ion . (1989) . Repor t on the Nat iona lSc ience Foundat ion d i sc ip l inary workshops on undergradua teeducation. W a s h i n g t o n , D C : A u t h o r .


11/11

EVALUATION OF URE 201National Science Foundation. (1996). Shaping the future: Ne wexpectations fo r undergrad uate education in science, mathem at-ics, engineering, a nd techn olog y (NSF Publication No. 96-139).Arling ton, VA: Author.Ormro d, J. E. (1999). H uman lear ning (3rd ed.). Englew ood Cliffs,NJ: Prentice Hall.Richmond, G. (1998). Scientific apprenticeship and the role ofpublic schools: General education of a better kind. Journal ofResearch in Sc ience Teaching, 35, 583-587.Rosser, S. V. (1990). Female-friendly science: Applying women sstudy methods and theories to attract students. New York:Pergamon Press.Sadker, D., Sadker, M. (1985). Is the ok class room ok? Phi DeltaKappan, 55, 358-361.Seago J. L. Jr. (1992). The role of research in undergraduateinstruction. The American Biology Teacher, 54, 401--405.Seymo ur, E., Hewitt, N. M. (1997). Talking about leaving: Whyundergraduates leave the sciences. Boulder, CO: WestviewPress.

Shodell , M. (1995). The question-driven classroom: Student ques-tions as course curriculum on biology. The American BiologyTeacher, 57, 278-281.Shuell, T. J. (1997). Teaching and learning in a classroom context.In D. C. Berline r R. C. Calfee (Eds.), The handbook ofeducational psychology (pp. 726-764). London: Prentice Hall.Shyman sky, J., Kyle, W. (1992). Establis hing a research agenda:Critical issues of science curriculum reform. Journal of Re-search in Scien ce Teaching, 29, 749-778.Spilich, G. (1997). Does undergraduate research pay off?. Councilon Undergraduate Research Quarterly, 18, 57-59, 89.Tobias, S. (1990). They re not dumb, they re different: Stalking thesecon d tier. Tucson, AZ: Research C orporation.Tobias, S. (1993). Overcoming math anxiety: Revised and ex-panded. New York: Norton.

Re c e i ve d June 3 , 1999Re v i s ion r e c e ive d Se p te mbe r 8 , 1999

A c c e p te d Se p te m be r 8, 1999

A M E R I C A N P S Y C H O L O G I C A L A S S O C I A T I O NS U B S C R I P T I O N C L A I M S I N F O R M A T I O N Today s Date:

We provide th is form to assis t members, inst i tu t ions, and nonmem ber indiv idua ls wi th any subscr ip t ion problems. With theappropriate nformat ion we can begin a resolu t ion . I f you use the se rv ices of an agent , p lease do NO T dupl ica te c la ims throughthe m a nd d i r ec t ly t o u s . PLEA SE PRI N T CLE A RL Y A N D IN IN K IF PO SSIBL E.

PRINT FULL NAME OR KE Y NAME OF INSTITUTION

ADDRESS

CITY ffrA ~UNTR Y ZIP

YOUR NAME AND PHONE NUMBER

T I T L E

MEMBEROR CUSTOMERNUMBER MAY BE FOUND ON ANY PASTISSUE LABEL)

DA TE YOU R OR DE R WA S MAll IRI~OR PHONED)PREPAID CHECK CHARGE

CHECK/CARDCLEARE D DATE:.If po~ible, send a copy, fre t and back, of your ~u~f lled c he ~ to help us in our ref~4u~ho your claim.)

ISSUES: -- MISSING -- DAMAGED

VOLUME OR YEAR NUMBEROR MONTH

Th an k you. On ce a claim is received nd resolve~ delivery f replacement issues outinely akes 4 6 eeks.TO BE FILLED OU T BY AP A STAFF)

DATE RECEIVED: DATE OF ACTION:ACTION TAKEN: INV. NO. DATE:STAFF NAME: LABEL NO. DATE:

end this for m to APA Subsc ription Claim s, 750 First Street, NE, Washin gton, De 20002.4242P L E A S E D O N O T R E M O V E . A P H O T O C O P Y M A Y B E U S E D .

Evaluation of an Undergraduate Research Experience

Documents

Transcript of Evaluation of an Undergraduate Research Experience