This article was downloaded by: [Washington State University Libraries ]On: 24 October 2014, At: 21:03Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Journal of Continuing HigherEducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/ujch20

Measuring Students’ Engagement onCollege Campuses: Is the NSSE anAppropriate Measure of Adult Students’Engagement?Karen Price a & Sandra Nicks Baker aa Belmont Abbey College , Belmont , NCPublished online: 29 Feb 2012.

To cite this article: Karen Price & Sandra Nicks Baker (2012) Measuring Students’ Engagement onCollege Campuses: Is the NSSE an Appropriate Measure of Adult Students’ Engagement?, The Journalof Continuing Higher Education, 60:1, 20-32, DOI: 10.1080/07377363.2012.649127

To link to this article: http://dx.doi.org/10.1080/07377363.2012.649127

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Karen Price is an Assessment and Research Analyst and Sandra Nicks Baker is the Director of Institutional Research at Belmont Abbey College in Belmont, NC.Address correspondence to Karen Price, Assessment and Research Analyst, Belmont Abbey College, 100 Belmont–Mt. Holly Road, Belmont, NC 28012, USA (E-mail: Karenprice@bac.edu).

The Journal of Continuing Higher Education, 60:20–32, 2012Copyright © 2012, Association for Con tinu ing Higher EducationISSN 0737-7363DOI: 10.1080/07377363.2012.649127

Introduction

BackgroundAs institutions seek to promote student engagement

on campus, the National Survey of Student Engagement (NSSE) has become a measure commonly used to docu-ment how the institutions are meeting educational goals and providing a measure of student learning outcomes. The NSSE is an annual survey that collects information from fi rst-year and graduating senior students regarding their undergraduate educational experiences. The NSSE assesses the extent to which students engage in a variety of activities that are empirically related to desirable college outcomes including student learning and development, students’ satisfaction with the institution (sometimes used a proxy

for perceived quality of the institution), persistence, and retention (Kuh, 2009). Knowledge of and recognition of factors that contribute to the students’ sense of belonging, feeling of involvement, perception of a supportive campus atmosphere, and conviction of scholarly personal growth enable an institution of higher education to deliberately create an environment conducive to academic and social integration and to respond to needs of students. NSSE has gained national recognition as a viable assessment measure.

Since 2000 more than 1,400 institutions and 2.7 million students have taken part in the NSSE. In addition to providing participating institutions information on the quality of student engagement on their individual campuses, the survey is being used to rate institutions on various

Measuring Students’ Engagement on College Campuses: Is the NSSE an Appropriate Measure of Adult Students’ Engagement?Karen PriceSandra Nicks Baker

Abstract. As institutions seek to promote student engagement, the National Survey of Student Engagement has become a measure commonly used to document how institutions are meeting educational goals, but there is some question as to its applicability for certain undergraduate populations. 2010 survey results were analyzed for 125 adult and 69 traditional-age seniors attending a small, private, four-year institution in the U.S. South to evaluate differences between the two groups among 42 items composing fi ve benchmark areas. Results provided support for the hypothesis that adults would score lower than traditional-age students on survey items that are more applicable to traditional students because of their focus on out-of-classroom experiences. Findings of this study have implications for the use of National Survey of Student Engagement benchmarks in assessing adult student engagement.

Keywords. adult college students; nontraditional students; NSSE; student engagement

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

The Journal of Continuing Higher Education • 21

aspects of collegiate experiences for the general public. For example, both USA Today and the Voluntary System of Accountability offer Web sites that provide the public with information on student engagement. Using the NSSE, institutions have begun to assess engagement in a more intentional and empirical way.

While NSSE results are being used on a more fre-quent basis by institutions in program planning and as evidence for student learning in accreditation reviews, some researchers question the applicability of the results to certain undergraduate populations (Lerer & Talley, 2010). For example, the work of Lerer and Talley (2010) found that students’ traditional or nontraditional status was a signifi cant predictor for three of the NSSE benchmarks showing support for their argument that the benchmarks contain items that are primarily applicable to traditional students. In addition, there has been some question as to the validity and stability of the factor structure of the NSSE (LaNasa, Cabrera, & Trangsrud, 2009). LaNasa and col-leagues (2009) suggest that student engagement may be “separate and distinct from the fi ve benchmark descriptions utilized by NSSE researchers as the primary means to share results and compare institutions” (p. 328). Data provided by NSSE within their psychometric portfolio show only three of the fi ve NSSE benchmarks demonstrate high internal consistency, and the report cautions, “Results suggest that these two benchmarks [Active and Collaborative Learning (ACL) and Enriching Educational Experiences (EEE)] would be considered less reliable for use in statistical analyses” (NSSE, 2011, p. 3).

NSSE is predicated on the assumption that certain student behaviors are indicators of students’ engagement in the learning process. The NSSE was constructed based on Chickering and Gamson’s (1987) Principles of Good Practice for Undergraduate Education, which outlined seven categories of effective educational practice that infl uence the quality of students’ educational experiences. These areas of effective practice include student–faculty contact, cooperation among students, active learning, prompt feedback, time on task, high expectations, and respect for diverse talents and ways of learning. Gener-ally speaking, the more students engage in these kinds of activities, the more they learn and the more likely they are to persist and graduate from college (Kuh, Kinzie, Buckley, Bridges, and Hayek, 2006). Broadly defi ned, the concept of student engagement, the extent to which students take part in educationally effective practices, is the basis for the NSSE benchmarks. Another important premise within this framework is that institutional policies and practices infl uence levels of such engagement within a particular col-

lege or university (Pike & Kuh, 2005). The use of the term engagement in the NSSE is analogous to Astin’s (1984) term involvement in his original articulation of student involvement theory.

Theory of Student InvolvementAstin’s theory of student involvement (1984) posits that

an involved student is one who devotes considerable energy to academics, spends much time on campus, participates actively in student organizations and activities, and interacts often with faculty (Astin, 1984, p. 292). Astin’s theory of involvement focuses on behaviors in which students engage in traditional forms of academic and social educational ex-periences. The most persuasive types of involvement “turn out to be academic involvement, involvement with faculty, and involvement with student peer groups” (Astin, 1996, p. 126). Astin defi ned involvement to mean the student’s in-vestment of “physical and psychological energy”—time and effort—in the academic enterprise (Astin, 1999; Davis & Murrell, 1993), with persistence becoming the by-product of student involvement. Student attrition was viewed by Astin as the absolute form of noninvolvement.

Interestingly, although Astin (1984) defi ned student involvement as “the amount of physical and psychological energy that the student devotes to the academic experi-ence” (p. 298), his focus is primarily on the physical behaviors in which the student engages: participating in campus organizations, interacting with faculty and peers, attending campus events such as athletics competitions and cultural performances, and spending time studying, for example. He emphasizes a purposeful decision to focus on the behavioral components of involvement rather than motivational components in making the distinction that “it is not so much what the individual thinks or feels, but what the individual does, how he or she behaves, that defi nes and identifi es involvement” (p. 298). In his model, Astin (1999) discussed how characteristics of the student interact with the environment to impact the outcomes for the student. The quality and quantity of the student’s involvement is directly related to the developmental growth a student experiences. Numerous studies have provided evidence to support Astin’s (1999) theory that students’ level of involvement in college and the nature of their interactions with the environment are related to their overall educational growth.

The path to successful educational outcomes is a complex one. Using Astin’s theory of involvement as a conceptual framework, student engagement represents the intersection between the students’ behaviors, the time and effort students put into their work, peer involvement and interaction with faculty, and the institutional conditions

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

22 • NSSE and Adult Students’ Engagement

that support students’ success. Institutional assessment of student engagement is done, in part, because institutions recognize that it represents the aspects of students’ behavior and institutional fulfi llment of its commitment to student success that colleges and universities can impact as op-posed to the precollege characteristics and experiences of the student that are beyond the immediate control of the college of university (Kuh et al., 2006).

Engagement of Adult StudentsAccording to the National Center for Education

Statistics (2009), adult students 25 years of age and older make up almost half of all enrollment in degree granting institutions. Yet, adult students in higher education are often viewed through a “defi cit lens” where institutions look to precollege characteristics and being academically underprepared as reasons for attrition (Isserlis, 2008). Although colleges are becoming more responsive to the educational needs of adult learners, Sissel, Hansman, and Kasworm (2001) commented:

Because higher education has been anchored in its historical traditions of residential, selective education and because it is based in perceptions of a privileged place and role for young adult leadership development, this environment em-braces full-time, residential youth. Little space, voice, and value are given to other groups and in particular those who are the most different from young students: adult learners. (p. 18)

Nontraditional-aged students often do not live on campus and generally have other priorities such as work and family, which can limit their availability to participate in many common forms of campus engagement. Rather than viewing adult students as “fragmented learners who cannot devote suffi cient time, energy, and resources to intellectual engagement,” campuses need to embrace the ideals of valuing family, work, and making connections within the curricular experience (Sissel et al., 2001, p. 25). Research has shown that the widely accepted mod-els of student integration often fall short when applied to diverse student populations (Pascarella, Duby, & Iverson, 1983). Astin’s theory of student involvement, for example, fails to address what involvement may mean to the adult learner who generally commutes to campus, attends in the evening or weekends, and works at least part-time (Chaves, 2006).

For adult students, there appears to be a differentiation between participation behavior (involvement/engagement)

and a psychological sense of integration. This distinction was highlighted over a decade ago by Hurtado and Carter (1997) who suggested that “integration can mean some-thing completely different to student groups who have been historically marginalized in higher education.” More recently, Bean (2005) noted that Astin’s view of involvement solely as behavior does not provide a complete picture of student engagement; while the behavioral component is necessary, it is not a suffi cient conceptualization of engage-ment. As he notes, “Participating in events without commit-ting psychological energy to them indicates that they are unimportant to the student and thus ineffectual in changing the student. . . . Behavior without thought is not likely to lead to the gains associated with engagement” (pp. 2–3). Thus, the model of student engagement being assessed by NSSE may fail to account for the psychological components of engagement in learning. This may be particularly important as we investigate adult students’ engagement. Donaldson and Graham (1999) stated:

Despite a lack of certain types of campus in-volvement and recent academic experience, adult students apparently learn and grow as much or more than younger students during their undergraduate collegiate experiences. This implies that adults may be using different skills, techniques, settings, or interactions with faculty, fellow students, and others to achieve their desired results. (p. 26)

Adults often spend less time on campus engaging in traditional college activities due to external responsibilities such as work and family competing for their time (Choy, 2002); therefore, the classroom becomes an integral component of their learning experiences. According to Donaldson and Graham’s (1999) model for adult student outcomes “the connecting classroom component addresses the ways that adults use the classroom and their interac-tions with students and faculty as a springboard for their learning” (p. 28). Kasworm (2003) found that adults’ pat-terns of learning engagement were centered on knowledge acquisition that could transform real-life experience. In her analysis, Kasworm (2003) contended that the classroom was the defi ning college experience for adult students rather than the peer group or campus experience. Despite the fact that nontraditional students’ engagement in college activities is often divergent from traditional-age students, researchers have demonstrated that nontraditional students perform academically as well or better than younger, more traditional students (Carney-Crompton & Tan, 2002;

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

The Journal of Continuing Higher Education • 23

Donaldson & Graham, 1999; Graham & Donaldson, 1999; Richardson & King, 1998).

Assessment of college students’ engagement as it relates to student learning outcomes often subscribes to the conventional defi nitions of campus involvement as it relates to co-curricular and extracurricular activities. Nontraditional students’ engagement may fall outside of that which is traditionally defi ned by the academic community. Assessment tools such as the NSSE may be inappropriate for the adult learner as a measure of a college’s ability to facilitate an environment conducive to students being suc-cessful. In fact, the basis for NSSE’s conceptual framework, Chickering and Gamson’s (1999) seven principles of good educational practice, warns that an institution’s ability to demonstrate good practice is dependent on the students and their circumstances.

Research QuestionsThe current study attempts to learn how items

composing the NSSE benchmark scores perform for nontraditional students to determine if the NSSE is an appropriate measure for adult learners’ engagement. In our study, student engagement of adult and traditional-age students is compared across the 42 items contained within the fi ve benchmark dimensions. It is expected that adults will score lower than traditional-age students on NSSE items that are more applicable to traditional students due to their focus on out-of-classroom or nonacademic experiences. The researchers hypothesize the following with respect to the 42 core survey items composing the NSSE benchmarks:

1. No significant differences between adult and traditional-age students on any of the items that compose the Level of Academic Challenge Benchmark (LAC) are expected due to the fact that these items are meant to measure academic and classroom activities. We presume that these items are applicable to all students.

2. In the Active and Collaborative Learning Benchmark (ACL), three of the seven items appear to be primarily applicable to traditional-age students because these items refer to activities outside the classroom setting (“worked with other students outside of class,” “tutored or taught other students,” and “participated in a community-based project as a part of a regular course”). Many adult students will be constrained from engaging in these activities due to family and work obligations; therefore, the researchers would expect adults to score lower on these items.

3. Four of the six items in the Student-Faculty Interaction Benchmark (SFI) also appear to be mostly applicable to traditional-age students. Three of the items (“discussed ideas from your readings or classes with faculty members outside of class,” “worked with faculty members on activities other than coursework,” and “worked with a faculty member on a research project”) require students to be available to interact with faculty members outside of class. In addition, an adult student is less likely to have “talked about career plans with a faculty member or advisor” because they have likely arrived at college with a clear career goal in mind such as obtaining a degree to advance in their current job.

4. In the Enriching Educational Experiences Benchmark (EEE), 5 of the 12 items seem to be limited to the experiences of traditional-age students, and adult students would be hypothesized to score lower than traditional students. Four of the items are related to out-of-classroom activities (participating in “internships and field experiences,” “community service or volunteer work,” “study abroad,” and “co-curricular activities”). The fi fth item (“participating in learning communities”) refers to an academic experience that is routinely designed for entering traditional students and not for mature adult students.

5. For the most part, the Supportive Campus Environment Benchmark (SCE) items appear to be applicable to both adult and traditional-age students. The one item that is an exception is the item that measures the extent to which the campus provides support for students to thrive socially. This particular item is most applicable to the traditional-age student who resides on-campus; therefore, we hypothesize adults would score lower on this item than traditional students.

Methods

ParticipantsData for this study were based on the analyses of the

2010 NSSE results for seniors attending a small, private, four-year institution in the U.S. South. The institution has a large adult student enrollment that composes half of the total student population. Most of the adult students attend classes in the evenings and weekends in the college’s adult degree program. Although a majority of adult students at this institution are classifi ed as enrolled full time, the students only enroll in two courses per eight week ses-sion and attend courses two nights per week. A majority of adult students in the sample work off campus, with

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

24 • NSSE and Adult Students’ Engagement

72% of the adult students working 30 or more hours per week. Primarily, traditional-age students enrolled at the college are residential students enrolled full-time. There were 194 seniors who completed the survey, representing a response rate of 46%, which is signifi cantly greater than the overall NSSE senior response rate of 33%, X2 = 32.45, p < .001. One-hundred and twenty-fi ve adult students and 69 traditional-age students responded to the survey. An adult student was defi ned as any student who was age 23 or older at the time of their fi rst enrollment at the college. Further descriptive characteristics of the two groups are described in Table 1.

MeasureThe NSSE is an annual national survey administered by

the Indiana University Center for Postsecondary Research to measure the extent to which students engage in education-ally purposeful activities that contribute to their learning and personal development (Kuh, 2003). In 2010, the self-report measure was administered to fi rst-year and senior students at 589 U.S. and Canadian institutions. Since its inception, more than 2.7 million students have completed the survey (NSSE, 2011). Based on student responses to 42 core survey items, the NSSE reports institutional scor-ing on fi ve benchmarks: (a) Level of Academic Challenge (LAC), (b) Active and Collaborative Learning (ACL), (c) Student-Faculty Interaction (SFI), (d) Enriching Educa-tional Experiences (EEE), and (e) Supportive Campus Environment (SCE). Descriptions of the fi ve benchmarks including reliability are included in Table 2.

Results

In order to evaluate the research hypothesis that nontraditional students would score lower than traditional-age students on 13 NSSE items that were judged to be not applicable to all students due to their focus on out of class or nonacademic experiences, t-tests for independent means were performed. In order to control for elevated type I error rate associated with running multiple t-tests,

a Bonferroni adjustment was utilized. Mean differences between adult and traditional students’ responses to all 42 core survey items contained within the NSSE benchmark dimensions were examined. Of the 42 core survey items included in the NSSE benchmark calculations, there were statistically signifi cant differences between traditional and nontraditional students’ responses on 20 items, almost half of the benchmark survey items. Of the 13 items that were hypothesized to have potential bias against adult students due to their focus on traditional students’ experiences, 9 items revealed statistically signifi cant differences, with adults scoring lower than traditional students on each (see Table 3).

Items contained within two NSSE benchmark areas, SFI and EEE, showed the greatest number of items where adult students scored signifi cantly lower than their traditional-age counterparts. Signifi cant differences between adult and traditional students’ answers to survey items also emerged for items that were hypothesized to be applicable to all students.

Items Composing the LAC Benchmark AreaWhile not hypothesized, results showed four survey

items where signifi cant differences emerged between adult and traditional students (see Table 4). Of the items in the LAC benchmark area with signifi cant differences, two items dealt with the number of written reports completed by the student. In both cases, adult students scored signifi cantly lower. Recognizing that this item appears to be applicable to all students, it may be useful for this institution to evaluate course and program requirements to determine if there is a difference in the focus on writing. The other two items where signifi cant differences emerged related to the amount of effort students applied to their academics. In both cases, adult students scored signifi cantly higher than traditional students on the amount of effort put forth to meet standards or expectations, t (196) = 3.746, p < .001, and the perceived emphasis the campus places on time spent studying, t (187) = 2.567, p < .01.

Items Composing the ACL Benchmark AreaWithin the ACL benchmark area, it was hypothesized

that adults would score lower than traditional students on three of the seven items in the benchmark. As hypothesized, adults did score signifi cantly lower on the item pertaining to tutoring or teaching other students, t (197) = –6.148, p < .001. However, they did not score signifi cantly dif-ferent than traditional students on the item “worked with classmates outside of class to prepare class assignments.” As shown in Table 4, the items related to participating in

Table 1. Descriptive Data of Participants in Study by Program.

Traditional-Age

Seniors

Adult

Seniors

Mean Age 21.4 Years 39.6 Years

Female/Male 38/28 96/23

Minority 21% 55%

Commuter 5 (10.42) 2 (12.50)

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

The Journal of Continuing Higher Education • 25

a community-based project as part of a regular course (service learning) approached significance. For the institution studied, this question may be less relevant to students’ experience due to the fact that there is no formal service learning offered at this college. Results showed both groups scoring low in this area. Interestingly, although not expected, adult students reported working with students on projects during class signifi cantly more than traditional students, t (201) = 4.026, p < .001.

Items Composing the SFI Benchmark AreaAs seen in Table 4, the adult seniors scored lower

than the traditional-age students on every item included in the SFI benchmark area except for item 1q, where both groups mean scores were the same. Those items within the SFI benchmark area that revealed signifi cant differences were all hypothesized to do so. For item 1q, it would not be expected to see students score differently as the item pertains to prompt feedback received from faculty on academic performance.

Further investigating item level means, independent samples t-tests revealed signifi cant differences between adult students and traditional students in mean item level responses within the SFI benchmark for three items, 1o,

1p, and 1s (as shown in Table 4). Each of these items where nontraditional students scored signifi cantly lower than traditional-age students would require the student to interact with faculty outside of the classroom experience or outside of course and program requirements. The items contained within the SFI benchmark focus almost entirely on students interacting with faculty outside of the classroom rather than as a part of the curricular experience.

Items Composing the EEE Benchmark AreaAnother area where a copious amount of items

with signifi cant differences between adult students and traditional-age students emerged was for items contained within the EEE benchmark area. Although the researchers hypothesized differences for only 5 of the 12 items within this benchmark, nontraditional-age students scored lower on average across all items except for two: items 1l and 10c (see Table 4). In investigating the mean differences between nontraditional adult students and traditional-age students for questions representing the EEE benchmark area, inde-pendent samples t-tests showed these mean differences to be signifi cant for all items except items 1u and 1l. These two items were not hypothesized to be different for the two groups. Item 1l pertains to the use of electronic medium

Table 2. Descriptions of NSSE Benchmark Scales.

Scale

Number of

Items

Senior

Cronbach’s � Description of Scale

LAC 11 .765 Measures the amount of time students spend in activities related to challenging intellectual and creative work, such as preparing for class, the amount of reading and writing performed, and the institutional expectations for academic performance.

ACL 7 .672 Measures the extent to which students engage in classroom activities and collaborate with others in solving problems inside and outside the classroom.

SFI 6 .740 Measures the extent to which students interact with faculty outside the classroom and the extent to which faculty give prompt feedback on academic performance.

EEE 12 .662 Measures complementary learning opportunities inside and outside the classroom that support academic programs. The scale includes items on the extent to which students interact with students of different ethnic and political backgrounds, their involvement in co-curricular activities, students’ use of information technology, and their participation in study abroad, internships, and community service activities.

SCE 6 .795 Measures the extent to which students feel that their peers, faculty members, and administrative personnel help them to succeed both academically and socially.

Note: Reliability information obtained from NSSE Psychometric Portfolio (2011).

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

26 • NSSE and Adult Students’ Engagement

to discuss or complete an assignment. In today’s society where the reliance on technology applications, e-mail, and social media dominate, it is not surprising that there were no signifi cant differences in students’ responses to this item. Although the traditional-age students have been exposed to technology throughout their lives, the adult students appear to embrace this technology for classroom and educational experiences to combat the fact that they are on campus less to interact with students and peers face to face. Item 1u asks students to respond to how often they had serious conversa-tions with students of a different race or ethnicity and was hypothesized as being applicable to all students.

Items Composing the SCE Benchmark AreaRecognizing that the development of quality relation-

ships with faculty, staff, and peers are equally important to adult students as traditional students, the researchers hypothesized that only one item contained within the SCE benchmark area would result in signifi cant differences. It was hypothesized that adult students would score lower than traditional students on the item asking students if the

campus environment provided them support they needed to thrive socially; however, no signifi cant differences emerged. As shown in Table 4, adult and traditional students viewed the campus environment similarly with respect to sup-porting their academic, social, and nonacademic family responsibilities. It is also worth noting that, in our sample, adult students rated their quality of relationships with administrative personnel and offi ces as signifi cantly more positive than traditional students, t (188) = 6.261, p < .001. This may be refl ective of an intentional organizational structure at this institution to serve adult students through a separate adult degree program offi ce where students have dedicated personnel to support them with multiple facets of the educational process.

Discussion

Results of the current investigation provided support for the hypothesis that adult students would score signifi -cantly different than traditional-age students on many core survey items within the NSSE benchmarks due to their being

Table 3. Signifi cant Differences for Items in the Five NSSE Benchmarks Considered Nonapplicable to All Students.

Item Benchmark p value

1h. Worked with classmates outside of class to prepare class assignments

ACL 0.131

1j. Tutored or taught other students ACL <0.001** T

1k. Participated in a community-based project as part of a regular course

ACL 0.057

1o. Talked about career plans with a faculty member or advisor SFI <0.001** T

1p. Discussed ideas from your readings or classes with faculty members outside of class

SFI <0.001** T

1s. Worked with faculty members on activities other than coursework

SFI <0.001** T

7a. Practicum, internship, fi eld experience, co-op experience, or clinical assignment

EEE <0.001** T

7b. Community service or volunteer work EEE <0.001** T

7c. Participated in a learning community EEE 0.015

7d. Worked on research project with a faculty member outside of course or program requirements

SFI <0.01** T

7f. Studied abroad EEE <0.01** T

9d. Hours per seven-day week spent participating in co-curricular activities

EEE <0.001** T

10e. Campus environment provided the support you needed to thrive socially

SCE 0.857

Note: * p < .01, ** p < .001, T traditional students signifi cantly higher.

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

The Journal of Continuing Higher Education • 27

Table 4. Descriptive Statistics for Item Level Responses in the Five Benchmark Areas.

Mean Composite Score

Adult Senior Traditional Senior t

LAC

1r. Worked harder than you thought you could to meet an instructor’s standards or expectations

3.21 2.76 3.746**

2b. Coursework emphasized analyzing the basic elements of an idea, experience, or theory

3.29 3.26 0.278

2c. Coursework emphasized synthesizing and organizing ideas, information, or experiences

3.12 3.11 0.045

2d. Coursework emphasized making judgments about the value of information, arguments, or methods

3.11 3.14 –0.257

2e. Coursework emphasized applying theories or concepts to practical problems or in new situations

3.27 3.21 0.473

3a. Number of assigned textbooks, books, or book-length packs of course readings

3.32 3.12 1.318

3c. Number of written papers or reports of 20 pages or more

1.46 1.66 –2.038

3d. Number of written papers or reports between 5 and 19 pages

2.29 2.73 –3.163*

3e. Number of written papers or reports fewer than 5 pages

2.63 3.09 –2.706*

9a. Time spent preparing for class 3.80 3.83 –0.095

10a. Campus environment emphasized spending signifi cant amounts of time studying and on academic work

3.34 3.06 2.567*

ACL

1a. Asked questions in class or contributed to class discussions

3.59 3.34 2.362

1b. Made a class presentation 3.08 2.80 2.367

1g. Worked with other students on projects during class

2.63 2.15 4.026**

1h. Worked with classmates outside of class to prepare class assignments

2.35 2.53 –1.516

1j. Tutored or taught other students 1.36 2.06 –6.148**

1k. Participated in a community-based project as part of a regular course

1.49 1.73 –1.912

1t. Discussed ideas from your readings or classes with others outside of class

3.09 3.12 –0.171

SFI

1n. Discussed grades or assignments with an instructor

3.00 3.11 –0.953

1o. Talked about career plans with a faculty member or advisor

2.30 2.93 –4.465**

1p. Discussed ideas from your readings or classes with faculty members outside of class

2.08 2.57 –3.614**

Continued

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

28 • NSSE and Adult Students’ Engagement

Mean Composite Score

Adult Senior Traditional Senior t

1s. Worked with faculty members on activities other than coursework

1.38 2.10 –6.097**

7d. Worked on research project with a faculty member outside of course or program requirements

1.96 2.30 –2.678*

EEE

1l. Used an electronic medium (listserv, chat group, instant messaging, etc.) to discuss or complete an assignment

2.86 2.59 1.776

1u. Had serious conversations with students of a different race or ethnicity

2.69 2.83 –1.015

1v. Had serious conversations with students who are very different from you in terms of religious beliefs, political opinions, or personal values

2.62 3.01 –2.825*

7a. Practicum, internship, field experience, co-op experience, or clinical assignment

2.92 3.45 –3.497**

7b. Community service or volunteer work 2.84 3.78 –6.327**

7c. Participated in a learning community 2.17 2.54 –2.456

7e. Foreign language coursework 2.12 2.65 –3.854**

7f. Studied abroad 1.98 2.23 –2.673*

7g. Independent study or self-assigned major 2.02 2.61 –4.582**

7h. Culminating senior experience 2.75 3.54 –5.490**

9d. Hours per seven-day week spent participating in co-curricular activities

1.20 3.30 –10.508**

10c. An institutional climate that encouraged contact among students from different economic, social, and racial or ethnic backgrounds

2.62 2.19 2.805*

SCE

8a. Quality of relationships with other students 5.77 5.71 0.302

8b. Quality of relationships with faculty members 5.64 5.81 –0.865

8c. Quality of relationships with administrative personnel and offi ces

5.59 4.15 6.261**

10b. Campus environment provided the support you needed to help you succeed academically

3.15 2.97 1.362

10d. Campus environment helped you cope with your nonacademic responsibilities (work, family, etc.)

2.16 2.13 0.148

10e. Campus environment provided the support you needed to thrive socially

2.07 2.09 –0.181

Note: * p < .01, ** p < .001.

primarily applicable to traditional students’ experiences. Adult students scored signifi cantly lower than traditional students on 20 of the core survey items which, according to NSSE, suggests that adult students are less engaged academi-cally and socially than their traditional-age counterparts. However, it is possible that these students simply engage

in the college experience differently than traditional stu-dents. This may be a function, in part, of the adult students’ motivation for pursuing higher education and/or how they create a sense of community. Adult students tend to be more concerned with gaining new skills and academic knowl-edge while traditional students may be more likely to view

Table 4. Continued.

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

The Journal of Continuing Higher Education • 29

college as a path toward a career and a means to build social relationships (Eppler, Carsen-Plentl, & Harju, 2000; Lerer & Talley, 2010; Werring, 1987). If adult students do, in fact, engage in the college experience differently, it brings into question the usefulness of the NSSE as a measure of adult students’ engagement.

The results of the current study also provide some ad-ditional support of the work of Lerer and Talley (2010) who also noted signifi cant differences between nontraditional and traditional students on many of the core survey items. In particular, there were numerous items found in the SFI and EEE benchmarks where signifi cant differences existed, which may suggest that these benchmark items may fail to refl ect the needs and expectations of the nontraditional student population. The current study found differences in many of the same items where Lerer and Talley (2010) noted signifi cant correlations between item responses and the traditional/ nontraditional variable. It appears that some benchmarks contain items that are applicable to all students, while items composing other benchmark dimen-sions may better refl ect a traditional college experience. The current study provides some evidence that adults may be more likely to integrate both socially and academically in the college through the curricular classroom experience.

For instance, Gilardi and Guglielmetti (2011) dem-onstrated that the relationship that adult learners have with faculty is a powerful infl uence on their academic experiences, but the relationship revolves around the teaching and learning process. It appears that the adult students in this study are engaged in the learning process for academic activities that occur within the classroom setting but differ from traditional students in the amount and type of interactions they have with faculty and peers outside of the classroom. Many of the core survey items appear to be more applicable to the traditional student as they pertain to out-of-classroom experiences atypical of adults’ college experience such as extracurricular experi-ences and supplemental curricular experiences such as study abroad, independent study, additional foreign lan-guage coursework, and community service and volunteer activities. The inclusion of so many items pertaining to out-of-classroom, nonacademic experiences makes one question if NSSE creators believe that they are a prerequisite condition to academic and social engagement. The focus on nonacademic experiences may potentially bias the items against adult students as these behaviors and experiences are generally viewed as more refl ective of the traditional students’ college experience.

For the adult students in this study, the classroom also seemed to serve as the platform for forming social

relationships with peers. For example, adult students were more likely than traditional students to indicate that they worked with other students during class, t (201) = 4.026, p < .001. Wyatt (2011) reported that nontraditional students did not feel the need to be engaged in specifi c activities or events on campus because their interaction and engagement with students was a result of classroom discus-sion or group activities in the classroom. In fact, Werring (1987) suggested that adult students actually prefer less formality in the classroom. Donaldson and Graham (1999) found that for part-time adult students it is these informal relationships developed in the classroom that help students make the connection between the curriculum and real life. In a study on adult undergraduates reentering higher educa-tion, Donaldson, Graham, Martindill, and Bradley (2000) demonstrated that the classroom served a pivotal role for developing relationships with other students “both before class, in class, during breaks, and after class” (p. 8). For adult students, structured curriculum activities and defi ned cohort-programs may help create perceptions of commu-nity for nontraditional students, promoting both academic and social involvement (Harris, 2007; Tinto, 1997). The results of the current study provide some support for the notion that adults may be creating informal learning communities within the “connecting classroom” and may be able to develop interpersonal relationships with other students through the classroom experience alone.

Limitations

The current research was exploratory in nature and cannot as a stand-alone study account for all of the complexities of the adult college experience, but the current study does support a call for additional research investigating the different ways in which adult students and traditional students engage socially and academically. The authors recognize that this study represents a small sample size and only includes one private school, thus limiting generalizability of fi ndings. In order to build a more mean-ingful sample size, future collaborative research allowing data to be compared across schools of different sizes and sectors would be benefi cial. In addition, the exploratory nature of the item level analysis helps to explain where differences exist, but it cannot fully elucidate the source of the invariance.

Conclusion

Although far from conclusive, there is some prelimi-nary evidence to suggest that adult students and traditional

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

30 • NSSE and Adult Students’ Engagement

students may engage in the college experience in different ways. To compensate for less time on campus, adults may utilize the classroom as the avenue for developing friend-ships and making meaning of their college experience. Adult students typically commute to campus, work at least part time, and often attend classes part time in the evenings or on weekends; therefore, it is important for colleges and universities to recognize the importance of the curricular experience as a means of engagement for these students.

It is important for continued research to focus on the distinct ways that nontraditional students engage and integrate in the college experience, including evaluating the appropriateness of our assessment tools and theoretical models for use with this population. In doing so, institutions can more accurately assess the experience of all college students rather than base the needs of our adult student population on models developed using traditional-age student models. In assessing effective practices in higher education, it may be necessary to delineate the behaviors and practices that are most applicable to all student populations and differentiate those from behaviors that are seemingly more often refl ected in the traditional students’ college experience. Within the current structure, it may be argued that the NSSE benchmark items may present some bias against adult students. As a case in point, NSSE has positioned itself as a measure of effective educational practices in their partnerships with accountability initiatives designed to provide accessible information to the public as a demonstration of stewardship with the inclusion of NSSE results in the Voluntary System of Accountability (VSA) College Portrait as well as organizations associated with college rankings such as USA Today and U.S. News and World Report reporting on NSSE benchmark scores. Further research needs to evaluate the appropriateness of this measure for institutions with large adult student popula-tions. The results of this study provide some preliminary evidence to support the notion that NSSE benchmarks may put colleges with large adult student populations at a disadvantage when results are used for accountability purposes.

Areas for further research in this area are plentiful. With the increasing numbers of adult students entering our colleges and universities, it is important to investigate which factors most impact how these students interact with peers, faculty, and the institution itself. For example, to tease out the role of commuter status on adult stu-dents’ engagement patterns, additional research should evaluate traditional and nontraditional commuter students’ engagement. It may also be useful to investigate the appli-cability of measures of engagement to other populations,

such as asynchronous online learners, another growing population in higher education. In addition, as we seek to further understand the sources of variability for different populations, it would be useful to investigate sources of measurement invariance within the constructs utilizing confi rmatory factor analysis or item-response theory models. To further delineate the effects of age on col-lege students’ engagement patterns, additional research utilizing structural equation modeling or path analysis techniques would allow for a more in-depth evaluation of mediating and moderating variables. Investigating student engagement within the academic curricular context rather than as a separate and distinct phenomenon may provide additional insight into adult engagement. As colleges and universities increasingly use student engagement and student satisfaction with the college experience as proxies for student learning outcomes for accountability purposes, particularly within accreditation reviews and in national surveys of institutional quality such as U.S. News and World Report (NSSE, 2005), it is important to recognize that one size may not fi t all and that measures of engagement may not adequately refl ect the experiences of certain populations.

References

Astin, A. W. (1984). Student involvement: A developmental theory for higher education. Journal of College Student Personnel, 25, 297–308.

Astin, A. W. (1996). Involvement in learning revisited: Lessons we have learned. Journal of College Student Development, 37, 123–134.

Astin, A. W. (1999). Student involvement: A developmental theory for higher education. Journal of College Student Development, 40, 518–529.

Bean, J. P. (2005). A conceptual model of college student engagement. Paper presented at the annual meeting of the Association for the Study of Higher Education, Philadelphia, Pennsylvania.

Carney-Crompton, S., & Tan, J. (2002). Support systems, psychological functioning, and academic performance of nontraditional female students. Adult Education Quarterly, 52(2), 140–154.

Chaves, C. (2006). Involvement, development, and retention: Theoretical foundations and potential extensions for adult community college students. Community College Review, 34(2), 139–152.

Chickering, A., & Gamson, Z. (1999). Development and adaptations of the seven principles for good practice

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

The Journal of Continuing Higher Education • 31

in undergraduate education. New Directions for Teaching and Learning, 80, 75–81.

Choy, S. (2002). Findings from the condition of education 2002: Nontraditional undergraduates [Brochure]. U. S. Department of Education, National Center for Education Statistics, NCES 2002-012. Retrieved from http://nces.ed.gov/pubs2002/2002012.pdf

Davis, T. M., & Murrell, P. H. (1993). Turning teaching into learning: The role of student responsibility in the collegiate experience. ASHE-ERIC Higher Education Report No. 8. Washington, DC: The George Washington University, School of Education and Human Development.

Donaldson, J. F., & Graham, S. W. (1999). A model of college outcomes for adults. Adult Education Quarterly, 50(1), 24–40.

Donaldson, J. F., Graham, S. W., Martindill, W., & Bradley, S. (2000). Adult undergraduate students: How do they defi ne their experiences and success? The Journal of Continuing Higher Education, 48(2), 2–11.

Eppler, M. A., Carsen-Plentl, C., & Harju, B. L. (2000). Achievement goals, failure attributions, an academic performance in nontraditional and traditional college students. Journal of Social Behavior and Personality, 15(3), 353–372.

Gilardi, S., & Guglielmetti, C. (2011). University life of non-traditional students: Engagement styles and impact on attrition. The Journal of Higher Education, 82(1), 33–53.

Graham, S., & Donaldson, J. (1999). Adult students’ academic and intellectual development in college. Adult Education Quarterly, 49(3), 147–162.

Harris, B. A. (2007). The importance of creating a “sense of community” within an adult student cohort. Journal of College Student Retention: Research, Theory & Practice, 8(1), 83–105.

Hurtado, S., & Carter, D. (1997). Effects of college transition and perceptions of the campus racial climate on Latino college students’ sense of belonging. Sociology of Education, 70, 324–345.

Isserlis, J. (2008). Adults in programs for the “academically underprepared.” New Directions for Adult and Continuing Education, 120, 19–26.

Kasworm, C. (2003). Adult meaning making in the undergraduate classroom. Adult Education Quarterly, 53(2), 81–98.

Kuh, G. (2003). What we’re learning about student engagement from NSSE. Change, 35(2), 24–32.

Kuh, G. D. (2009). The National Survey of Student Engagement: Conceptual and empirical foundations. New Directions for Institutional Research, 141, 5–20.

Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2006). What matters to student success: A review of literature. Commissioned Report for the National Symposium on Postsecondary Student Success: Spearheading a Dialog on Student Success. National Postsecondary Education Cooperative.

LaNasa, S. M., Cabrera, A. F., & Trangsrud, H. (2009). The construct validity of student engagement: A confi rmatory factor analysis approach. Research in Higher Education, 50, 315–332.

Lerer, N., & Talley, K. (2010). National Survey of Student Engagement’s (NSSE) benchmarks—One size fi ts all? On the Horizon, 18(4), 355–363.

National Center for Education Statistics. (2009). Table 201. Total fall enrollment in degree-granting institutions by control and type of institution, age, and attendance status of student: 2009. Digest of Education Statistics. Retrieved from http://nces.ed.gov/programs/digest/d10/tables/dt10_201.asp

National Survey of Student Engagement. (2005). Student engagement: Exploring different dimensions of student engagement. Bloomington, IN: Indiana University Center for Postsecondary Research.

National Survey of Student Engagement. (2011). About NSSE. Bloomington, IN: Center for Postsecondary Research, Indiana University, School of Education. Retrieved from http://nsse.iub.edu/html/about.cfm

National Survey of Student Engagement Psychometric Portfolio. (2011). Bloomington, IN: Center for Postsecondary Research, Indiana University, School of Education. Retrieved from http://www.nsse.iub.edu/links/psychometric_portfolio

Pascarella, E., Duby, P., & Iverson, B. (1983). A test and reconceptualization of a theoretical model of college withdrawal in a commuter institution setting. Sociology of Education, 56, 88–100.

Pike, G. R., & Kuh, G. D. (2005). A typology of student engagement for American colleges and Universities. Research in Higher Education, 46(2), 185–209.

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4

32 • NSSE and Adult Students’ Engagement

Richardson, J., & King, E. (1998). Adult students in higher education: Burden or boon? The Journal of Higher Education, 69(1), 65–88.

Sissel, P. A., Hansman, C. A., Kasworm, C. E. (2001). The politics of neglect: Adult learners in higher education. New Directions for Adult and Continuing Education, 91, 17–27.

Tinto, V. (1997). Classrooms as communities: Exploring the educational character of student

persistence. Journal of Higher Education, 68(6), 599–623.

Werring, C. J. (1987). Responding to the older aged full-time student: Preferences for undergraduate education. The College Student Affairs Journal, 1, 13–20.

Wyatt, L. G. (2011). Nontraditional student engagement: Increasing adult student success and retention. The Journal of Continuing Higher Education, 59, 10–20.

Dow

nloa

ded

by [

Was

hing

ton

Stat

e U

nive

rsity

Lib

rari

es ]

at 2

1:03

24

Oct

ober

201

4