A COMPARISON AMONG THREE EXPERIMENTAL EDUCATION PROGRAMS

Vernon Williams

ABSTRACT." The primary characteristics of the three programs are used to compare participants to each other and to a group of students in the regular curriculum: (1) development of logical thought, (2) broader educational development, and, (3) career development, as well as selected academic characteristics {grades academic aptitude). The programs and their evaluations are described. The outcomes claimed for each program occurred more frequently in that program than in the comparison groups. The absence of differences in grades, despite differences in academic ability and in extent of program involvement, is interpreted as further evidence of the insensitivity of grades. The question of generality of the results is explored briefly.

In 1970 the University of Nebraska initiated a new experimental college, named Centennial, in celebration of the University 's reaching its one hundredth year. The Centennial Educational Program is a residential unit integrating living and learning. Faculty offices and classrooms are located in the student residence. Interaction between students and faculty is high. The basic structure of the Centennial Program is the learning contract. Ei ther s tudent or faculty can initiate a course. A course may consist of as few as two participants if both teacher and learner agree. Faculty also offer interdisciplinary courses, based on the common interests of the Fellows. Fellows serve two year terms, often part-time.

Participant self-governance is emphasized via group meetings, referenda, and other means. This and other program elements (student-faculty interaction, small group peer support, cultural participation, etc.) are aimed at facilitating student development, focusing on more than just classroom learning.

Community outreach and other forms of experiential learning are emphasized also in the areas of social science, humanities, and even in

Reprint requests should be addressed to Mr. Vernon Williams at The Universi ty of Nebraska, The Counseling Center, Seaton Hall, Lincoln, Nebraska 68588.

Alternative Higher Education, Vol. 5(4), Summer 1981 0361-6851/81/1400-0219500.95 ©1981 Human Sciences Press

220 ALTERNATIVE HIGHER EDUCATION

the sciences. One example entails s tudents accompanying an art ist to work with prison inmates.

An example of the other primary mode of instruction, the interdisciplinary course, is "Confrontation: The American Myth vs. a Finite Future ." The course includes perspectives offered by an economist, a physicist, and a humanist on the clash between traditional American values and the increasingly obvious limitations on resources.

In the second half of the decade two other experimental programs were launched on a somewhat less grand scale than was the case with Centennial. One, A D A P T (Accenting Development of Abs t rac t Processes of Thought), introduced a curriculum for the freshman year based on Piagetian notions. The A D A P T program consists of courses continuing the entire freshman year in six disciplines. The content of these courses is closely interwoven and, in accord with the Piagetian notion, the courses are based on the s tudents ' pas t experience. The courses also make heavy use of Paiget 's concepts of peer interaction to facilitate development and concrete activities as a basis for developing abst ract thinking. Students carry out tasks in small groups working in labs, on campus and in the community. The program also uses extensively the learning cycle developed by Karplus (1974) to implement Piaget 's concept of self-regulation (1972). Karplus' learning cycle includes the three steps of exploration, invention, and application.

The cycle applies the self-regulation concept by disequilibrating the learner {or put t ing her/him off balance a bit} with the exploratory act ivi ty arousing her/his curiosity and leaving the person with a question to be resolved. The invention phase introduces a concept which can reduce the uncertainty and lead to increased understanding. In the final s tep the learner uses the concept in some area other than the one in which it was explored originally.

All courses begin with an exploration phase in which students are required to observe a behavior or phenomenon and collect data about that phenomenon. Following such exploration, the data are organized and are used to form hypotheses, to provide a basis for generalization to other topics. With the background provided by these exploratory and invention activities, students are prepared to relate the concrete knowledge they have acquired to more abstract findings in the field, and to apply their knowledge to other areas. For example, some of the other functional relationships in economics have been explored by having students receive yearly incomes in play dollars. They then allocate these dollars to various expenses, investments, etc. Looking at the

VERNON WILLIAMS 221

relationship between the way they have allocated money and the way various families and individuals in the society allocate money provides a basis for making more meaningful abstract concepts like price indices and inflation. (Toml/nson-Keasey & Eisert, 1978, p. 100)

The second program, Freshman Human Services, combines an introduction to the subject with an experiental element and a seminar on the politics of service institutions. The program is organized around the experiential component which entails placing students in community agencies. Students do whatever they can to be useful in exchange for learning about the agency. They meet with the instructor each week to discuss what they have learned. Through the weekly seminar and the agency experience the students develop beginning levels of skills useful in the human services. The introductory course provides a relatively straightforward introduction to the area. The political science seminar examines the dynamics and politics of bureaucracy in public agencies such as the ones in which the students are placed.

In addition, several means are used to build a sense of community. Students are encouraged to live together, work on projects jointly, prepare with each other. Students are placed together in an agency, when possible, so they can support each other and facilitate learning.

The core course in this program contains three foci: 1. career exploration, 2. investigation of major human service areas, and, 3. human relations skill development, The student begins by visiting a variety of human service agencies from which one is chosen for a semester-long experience. Then several career development activities are introduced: inventorying career interests, offering a career workshop, and discussing a videotape dramatizing career development issues. Self-assessment, skill development exercises, and individually contracted experiences are used to teach basic human relations skills.

Students work together in small groups to produce two reports: {1} a presentation to naive persons about the human services area in which they worked, and {2} a description of some campus resource useful to human service students.

A comparison group was included with each evaluation undertaken. The comparison group was used as a norm against which to judge differences occurring among the experimental groups being evaluated. While a general, unselected group of university students was needed to provide a somewhat typical comparison, some control for the level of motivation was deemed necessary. Since experimental program

222 A L T E R N A T I V E H I G H E R EDUCATION

participants volunteered, the comparison group was chosen from volunteers also. The intellectual and academic measures were completed by students who volunteered but were not selected for an experimental program. The career inventories were answered by students from a special section of the introductory education course employing an experiential project.

HYPOTHESES ABOUT THE PROGRAMS

Although all three programs might be considered experimental {and at the University of Nebraska they are so considered), each is somewhat different from the others.

Thus one would expect differences to occur when the three programs are assessed. The programs were examined in relation to the hypotheses about their essential character and aims. The ADAPT program was founded on the basis of the hypothesis that development can be facilitated along the Piagetian scale of increasing logical function. The Centennial Program was initiated from a broader developmental base, the notion that students can grow in extra- academic, as well as in academic ways when an intentional effort is made to integrate the two parts of their experience. The Human Services Program was aimed at career development within the area for which the program is named.

While academic ability and achievement were not included in the goals of the three programs, at least two of them deliberately took positions concerning student selection based on these factors. ADAPT sought systematically to include a range of abilities, to mirror the range found in the entering class. Centennial staff decided to select students at random, but only after considerable debate and with some sentiment favoring a more elitist selection. Human Services students were entirely self-selected.

Although academic performance was not an essential element in the programs described here, critics have charged that students might be disadvantaged by participation in experimental programs. If students in the programs examined here were so affected, the programs should be differentiated in terms of breadth or amount of student participation. ADAPT constitutes the entire freshman year curriculum for its participants. Centennial participation is limited to six hours per semester. Human Services consisted originally of seven hours one semester and three the next. {Currently it is contained

VERNON WILLIAMS 223

within a single course.) Thus if these programs place the student at a disadvantage, ADAPT students should perform more poorly after the first year than the other two groups.

MODES AND MEANS OF COMPARISON

A number of indices were used to assess the characteristics mentioned in the preceding section. Academic aptitude scores {American College Testing Program} were identified as indicators of academic ability in the three groups. The American College Testing Program administers one of the two major college entrance tests in the United States. The test consists of questions dealing with English, Mathematics, Natural Sciences, and Social Sciences. A "Composite" score is obtained also to represent a summary of results in the other four areas. Two measures of logical, critical, or (in Piagetian terms} formal operational thinking included: a

local instrument devised by Tomlinson-Keasey from a variety of Piagetian tasks {1972} and the Watson-Glaser Test of Critical Thinking {1971}. The Tomlinson-Keasey is a paper and pencil test composed of problems designed to assess basic qualities Piaget attributed to formal operational (or logical} thought, including the ability to: Ca) separate variables, (b) conduct unconfounded tests of the effect of individual variables, (c) generate hypotheses systematically, (d) perform proportional reasoning, (e) conduct critical tests of hypotheses, (f} demonstrate an understanding of probability and (g} determine the relationship between two variables. CAn example of a task testees are required to complete involves determining outcomes of combinations of genetic eye color characteristics.) The Watson- Glaser test requires the person to make inferences, deductions and interpretations and to evaluate assumptions and arguments in situations described in the question stems. College grades were viewed as indicants of academic achievement.

William Perry's scale of intellectual and ethical development was used to rate interviews with program participants. Perry's scale describes the progression found among college students from absolute, "dualistic" thinking through relativistic thought, to a commitment to ideas and values ultimately subject to reexamination, albeit held with some firmness. Perry's interview procedure was followed (1970}. Two graduate students achieved 80% agreement in rating the interviews.

224 ALTERNATIVE HIGHER EDUCATION

Finally two assessments of career att i tudes were made. A "Career Concerns Inventory" was adapted from a dissertation by Pinkney {1973}. This inventory was designed to assess the number of problems the student experiences in arriving at career decisions. Split-half reliability obtained with this instrument was .83. A "Career Misconceptions Inventory," devised locally, was designed to assess false assumptions about the career selection and development processes. The split-half reliability of this instrument is .82. The thinking assessments {Tomlinson-Keasey and Watson-Glaser and the Perry scale} were chosen originally for their value in the evaluation of the ADAPT Program. Because they were not judged relevant by administrators of all three programs, the instruments were not administered to students in all three. Results of these measures simply are reported for the groups who completed them.

With the exception of scholastic apti tude scores the information was obtained at the end of the spring semester of the freshman year. College grades were obtained for the two years following students ' participation in these programs. The career inventories were completed by Human Services students and the comparison group at the beginning of the year, as well as at the end.

All s tudents in the Human Service and comparison groups were included in the study. Data were collected from all the ADAPT students, except for the career inventories, which were completed by only ten students who participated in a career exploration seminar. A sample of approximately 15% of Centennial program participants was included also. Although those degrees of inclusion in the evaluation clearly are not what would be designed in a scientifically based, systematic evaluation process, they do afford some comparison among the educational programs being examined. Until more comprehensive data are available, the information presented here will provide at least a tentative characterization, clearly preferable to none at all.

COMPARISON RESULTS

Academic Ability

Even though the ADAPT group generally scored lowest, no statistically significant differences were obtained among the four groups when the ACT results were compared. Composite results are shown in Table 1. {Complete results of comparisons among all AC~ scores may be obtained from the author.}

VERNON WILLIAMS 225

Table 1. Statistical Comparisons Among the Groups' Composite Aptitude (ACT)

1975-76 1976-77

Grou E N M SD N M SD

ADAPT 27 21.24 4.94 31 20.22 4.70

Centennial 19 21.89 5.01 24 21.95 5.13

Human Services 21 19.23 1.04

Comparison 27 24.25 3.77 34 24.12 5.54

1975-76 1976-77

F = .]4 F = .21

n.s. n.s.

Academic Performance

No differences occurred either year among the cummulative grade averages of the three groups. These tests are depicted in Table 2.

Operational~Critical Thinking

ADAPT students performed better than Centennial and comparison students on both measures of logical or critical thought. This comparison is presented in Table 3.

Perry Scale

Statistically significant differences were obtained among the three groups assessed on this scale. Centennial students were judged to occupy higher positions on the Perry scale than were the comparison group, who in turn were placed above ADAPT students. Centennial students reached a position typifying relativistic thinking. ADAPT students were placed below the point indicating relativistic thinking. Half the comparison group achieved relativism;; half, less. These results are shown in Table 4.

226 ALTERNATIVE HIGHER EDUCATION

Table 2. Grade Comparisons Among the Three Groups

Group 1975-76 1976-77

Centennial (N=26) M = 3.05 SD = ,54 M = 3.19

ADAPT (N=I9) M = 3.01 SD = .65 M = 2.90

Human Services (N:I9) M = 3.19 SD = .19 M = 3.18

Comparison (N=24) M = 3,04 SD = .30 M = 3,07

8D : .40

SD = ,74

SD = . 76

SD = , 63

1975-76 1976-77

F = ,01 F = ,03

n.s. n.s.

Table 3. Comparison of ADAPT and Centennial Groups on Formal Operations (Tomlinson-Keasey} and Watson-Glaser Test of Critic~ Thinking

Comparison ADAPT Centennial

Test (n=31) (n=~2) (n=32) F P

Tomlinson-Keasey

Composite Score

M 44.50 52.34 36.97 13.07 <,01

SD 9.92 11.94 13.29

Watson-Glaser

Pre-test

M 74.54 63.60 73,75

SD 8.39 8.23 10.09

Post-test

M 74.97 73.24 74.00

SD 10.02 10.49 11.70

4.03 <.02

VERNON WILLIAMS 227

Table 4. Evaluation of Perry Scale Results in Three Groups, ADAPT, Centennial, and Comparison

Position Number Centennial Comparison ADAPT

5-6 7 0 0

4 6 5 6

2-3 4 5 ii

Chi Sq~tare = 40.39* p < .001

~ith Yates correction

Career Measures

Only the Human Services Program and the regular students were assessed at the beginning of the year, as well as at the end. Results of these comparisons show that while Human Services students became less concerned about career decisions over the course of the year, the other group did not change. No differences occurred among the four groups at year's end. Table 5 depicts this set of outcomes.

On the Career Misconceptions Inventory neither general university students nor Human Services students changed over the year. The four groups did not differ at year's end. These findings may be seen in Table 6.

DISCUSSION

In spite of the differences in selection procedures among the three programs no differences in academic ability were found among the three. While no direct evidence is known to suggest that Human Services participants are less able than the other two groups, less certain students have been found to achieve lower grades than those who are more nearly decided about academic and career directions.

Certainly the aptitude scores would not lead to the prediction that ADAPT students would do better than Centennial students. Thus the fact that ADAPT grades are no worse than Centennial grades is contrary to expectation, whether based on measured ability or on the quantity of supposed detrimental experience accrued in the program.

228 ALTERNATIVE HIGHER EDUCATION

Table 5. Comparison of Career Concerns Results Among the Three Groups

Group

ADAPT

Centennial

i. Human Services

2. Human Services

i. Comparison

2. Comparison

Human Services 1 vs Human Services 2

Comparison 1 vs Comparison 2

F =

ADAPT

Centennial

Human Services 2

Comparison 2

M SD N

23.00 2.14 I0

17.5 .76 6

24.73 1.05 34

21.27 .93 26

21.17 1.00 36

22.83 1.35 36

t = 2.45 p <.05

t = .99 n.s.

. 4 7 n . s .

Table 6. Comparison of Career Misconceptions Results Among the Three Groups

Group

ADAPT

Centennial

i. Human Services

2. Human Services

I. Comparison

2. Comparison

Human Services 1 vs Human Services 2

Comparison 1 vs Comparison 2

F =

ADAPT

Centennial

Human Services 2

Comparison 2

M SD N

32.30 .56 I0

30 1.97 6

29.52 .34 34

29.73 .47 26

28.69 .92 39

28.60 .47 35

t = .26 n.s.

t = .09 n.s.

.35 n.s.

VERNON WILLIAMS 229

The measures of thinking skills support the findings reported elsewhere (Tomlinson-Keasey & Eisert, 1977} that the ADAPT Program develops logical, critical thinking more than other curricula do. Results obtained with the more broadly oriented Perry Scale suggest that the Centennial Program affects broader development than is tapped by the thinking skill instruments. Centennial students appear to exceed the other two groups on the dimensions of emotion and morality in addition to the logical element found in Perry's scheme. The Human Services Program appears to have been effective in reducing career concerns.

So what do these results say about experimental learning programs? The three programs studied here seem largely to have accomplished their major goals. The question arises whether all three goals can be accomplished in the same curriculum or whether a program must concentrate only on the intellectual, the career, or the broader emotional and moral goals.

Perhaps it is not surprising that s tudents in each experimental program performed best in areas emphasized by that program. Consider that both outcomes are possible. The programs reported here could have failed to attain their goals. What seems noteworthy is that each program accomplished more in the areas designated than did the others. The point is that when each characteristic measured here was central to a program's purpose, the characteristic was found more widely among the program's participants than among participants in programs with different goals.

While critics may charge that programs with different goals should not be compared, the rationale for this argument remains obscure. Programs not aimed at the goal being evaluated can with one exception be considered equivalent to a standard curriculum in so far as that goal is concerned. A major difference between an experimental program and a standard comparison group lies in its relation to the so- called Hawthorne effect. As most sophisticated readers of social science know, this effect refers to a nonspecific positive orientation toward the project in question. Comparison among experimental programs can control the Hawthorne effect because each program generates nonspecific excitement and positive expectation, quite apart from any goals-related effects. Thus results which are specified to the goal being evaluated and which are obtained in one program and not in another constitute strong evidence for goal attainment.

A final consideration regarding goal achievement is that in no instance did the standard comparison group perform better in relation to a goal than an experimental group. Taken together with the evidence cited already, this result affords a very robust argument for the validity of the experimental programs' hypothesized strengths.

230 ALTERNATIVE HIGHER EDUCATION

Finally, it is important to note that even though no differences were discovered in the grades earned by ADAPT and Centennial students, differences did occur in several indices of intellectual functioning. These findings support the oft-made observation that grades do not reflect important differences in thought.

REFERENCES

Karplus, R. Science curriculum improvement study: Teacher's handbook. Berkeley, California: Lawrence Hall of Science, 1974.

Perry, W. Forms of intellectual and ethical development during the college years: A scheme. New York: Holt, Rinehart and Winston, 1970.

Piaget, J. Intellectual evolution from adolescence to adulthood. Human Developmen~ 1972, 15 (1).

Pinkney, J.A. Comparison of structured and nonstructured group and individual vocational counseling using client satisfaction and an individualized measure of counseling effectiveness. Unpublished Doctoral Dissertation, University of Iowa, 1973.

Tomlinson-Keasey C. Formal operations in women from eleven to fifty-four years of age. Developmental Psychology, 1972, 6, 364.

Tomlinson-Keasey C. & Eisert, D. Second year evaluation of the ADAPT program. In Multidisciplinary Piagetian-Based Programs for College Freshmen. Lincoln: ADAPT, 1977, 120-133.

Tomlinson-Keasey C. & Eisert, D. Can doing promote learning in the college classroom? Journal of College Student Personne~ 1978, 19, 99-105.

Watson, G. & Glaser, E.M. Critical thinking appraisal. New York: Harcourt Brace & World, 1971.