Does Asking Pertinent Non-Academic Questions Make You a Better Researcher

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD

DOES ASKING PERTINENT NON-ACADEMIC QUESTIONS

MAKE YOU A BETTER RESEARCHER?

THE LINK BETWEEN NON-ACADEMIC AND

RESEARCH QUESTIONS

No part of this digital document may be reproduced, stored in a retrieval system or transmitted commercially in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.


Additional E-books in this series can be found on Nova’s website

under the E-book tab.


DOES ASKING PERTINENT NON-ACADEMIC QUESTIONS

MAKE YOU A BETTER RESEARCHER?

THE LINK BETWEEN NON-ACADEMIC AND

RESEARCH QUESTIONS

MARÍA ANTONIA PADILLA VARGAS UNIVERSIDAD DE GUADALAJARA

JAIRO TAMAYO UNIVERSIDAD VERACRUZANA

AND MARINA L. GONZÁLEZ-TORRES

UNIVERSIDAD AUTÓNOMA DE AGUASCALIENTES

New York

Copyright © 2013 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com

NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. Additional color graphics may be available in the e-book version of this book. LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA ISBN: 978-1-62948-587-4

Published by Nova Science Publishers, Inc. † New York

CONTENTS

Preface ix

Acknowledgment xi

About the Author xiii

Introduction xv

Chapter 1 Evaluating of the Relationship between Academic and Non-Academic Questions 1

Chapter 2 Effects of Exposure to Different Achievement Criteria on the Elaboration of Non-Academic Questions 17

Chapter 3 Variables That Affect the Process of Asking and Justifying Research Questions 35

General Conclusion 47

Index 51

Approved for publication by the members of the Editorial Board of

the Centro Universitario de Ciencias Biológicas y Agropecuarias, in the capacity of a Scientific Committee, according to the Protocol CINV.073/13.

Translated by: Cristiano Valerio dos Santos

PREFACE Answering the question of which criteria is best to identify a good

researcher is highly complex owing to the broad range of factors involved in the scientific practice. Some criteria are institutional, while others may be specific to a scientific community or even personal. On the one hand, some may argue that being capable of publishing autonomously (or at least independently of advisers) and of posing novel, useful, viable, and pertinent research questions is the main criteria by which a researcher should be judged. On the other hand, because publications may result in financial benefits to the author, they are not a good criterion to assess the academic value of a researcher. We do know however that academic training has not received sufficient attention by psychologists. Investigation on this topic has been conducted mainly by sociologists and educators, using ethnographic and descriptive methodologies. Psychological studies are scarce, fragmented and unconnected.

Given the importance of producing and spreading knowledge, of conducting novel research, and of training novice researchers to replace retiring academics (in Mexico, at least, the average active scientist is roughly 47 years old), it becomes important to analyze at least one element that might make a good researcher: posing novel, pertinent questions derived from a specific line of research. While in the process of analyzing this topic, the relationship between the ability to pose research questions and the skills involved in posing non-academic questions became apparent, and received a more in-depth analysis. We tried to sum up the results of these investigations and their conclusions in this book.

ACKNOWLEDGMENT To the Academic Workers Union at the University of Guadalajara

for providing the financial aid needed to print this book.

ABOUT THE AUTHOR María Antonia Padilla Vargas is a tenured professor at the University

of Guadalajara, working at the Center for Behavioral Studies and Research (CEIC). She is a member of National Researchers Association (SNI) and founded in 2006 the Mexican Association for Psychological Research (SMIP), over which she has presided ever since. She has published five books, 15 book chapters, and 26 research articles in indexed international journals on topics related to scientific practice, training new scientists, scientists interactions, and complex learning.

INTRODUCTION Answering the question of which criteria are best to identify a good

researcher is highly complex owing to the broad range of factor involved in the scientific practice. Some criteria are institutional, while others may be specific to a scientific community or even personal. On the one hand, some may argue that being capable of publishing autonomously (or at least independently of advisers) and of posing novel, useful, viable, and pertinent research questions are the main criteria by which a researcher should be judged. On the other hand, because publications may result in financial benefits to the author, they are not a good criterion to assess the academic value of a researcher. We do know however that academic training has not received sufficient attention by psychologists. Investigation on this topic has been conducted mainly by sociologists and educators, using ethnographic and descriptive methodologies. Psychological studies are scarce, fragmented and unconnected.

Given the importance of producing and spreading knowledge, of conducting novel research, and of training novice researchers to replace retiring academics (in Mexico, at least, the average active scientist is roughly 47 years old), it becomes important to analyze at least one element that might make a good researcher: posing novel, pertinent questions derived from a specific line of research. While in the process of analyzing this topic, the relationship between the ability to pose research questions and the skills involved in posing non-academic questions became apparent, and received a more in-depth analysis. We tried to sum up the results of these investigations and their conclusions in this book.

Three chapters comprise this book. In the first, we describe two studies that were carried out with high school students. We evaluated whether high school students were able to pose novel research questions

M. A. Padilla Vargas, J. Tamayo and M. L. González-Torres xvi

based on technical material they had previously read (Experiment 1) or whether they were able to pose non-academic questions without having to master a particular line of research (Experiment 2). Results showed that performance was poor with both types of questions and suggested that we might need to take a step back and analyze more basic skills such as reading comprehension and writing before exploring research questions per se.

Given these results, in another study, described in Chapter 2, we provided participants with differential information regarding the achievement criteria (defined as the criteria a particular performance must meet or to which it must adjust) when posing non-academic questions, that is, related to daily life, for which no specific training is required. Because high school students did not seem to be the most appropriate population for the study of academic and no-academic questions, we conducted this second experiment with high school, undergraduate, and graduate students and found that more specific achievement criteria facilitated the production of a greater number of questions, which also were more pertinent to the daily situations we presented. Higher-level students performed better and high-school students again were not capable of posing non-academic questions, even when they were given a specific achievement criterion.

Chapter 3 describes a series of studies, conducted with undergraduate and graduate students and experienced researchers, whose objective was to evaluate explicitly the variables that affect the production of research questions. The standard experimental protocol consisted of two groups (experimental and control), which were exposed to a pre-assessment, two exercises with or without corrective training, and a post-assessment (control groups were not given any training). Participants were required to derive a novel and pertinent research question from a series of empirical articles and justify it properly (pre- and post-assessments), to identify some of their elements (Exercise 1), or elaborate on them (Exercise 2). The empirical articles could either be used as they were published or be modified in some of their elements to make them “unstructured”, so we could evaluate whether this changes affected performance. Results of the studies conducted so far have shown that the performance of undergraduate and graduate students initially is poor, but with adequate training, especially corrective, it improves substantially from pre- to post-assessments. Experienced researchers

Introduction xvii

perform well from the start and their performance is not affected by how unstructured the texts are, contrary to what happened with the students.

The most remarkable finding to us is that both undergraduate and graduate students have poor reading and writing skills. Most are not used to reading technical materials, and find it hard to interact with them (even with corrective training, some are not able to obtain more than 60% of correct answers). Because research questions are derived from the comprehensive reading of empirical articles, it is understandable that this task can be arduous and sometimes impossible for them. Moreover, when required to justify the proposed research question, which implies writing skills, an intimate relationship between reading and writing is observed: 1) skilled readers turn out to be skilled writers and vice versa, and 2) improvements in one skill, brought about by training, relate to improvements in the other skill.

With this book, we wish to help training more and better researchers. Also, by presenting a detailed description of our methodology, we wish to propose a way to study this topic experimentally, so we can gather systematic data on the variables the affect the performance of researchers, both novice and expert.

Chapter 1

EVALUATING OF THE RELATIONSHIP BETWEEN ACADEMIC AND

NON-ACADEMIC QUESTIONS1 The main objective of science education is to train novice

researchers to be able to work independently from their advisers and teachers (Sánchez Puentes, 2004), carrying out novel investigations that are pertinent to the line of research under which they were trained (De La Fuente, Justicia, Casanova & Trianes, 2005). This must include the exercise of reading and writing skills, given that writing specialized materials is essential to the scientific practice (Cassany, 2006). Not only does this activity permit the diffusion of ideas and empirical evidence, it also promotes the generation of new ideas (Keys, Hand, Prain & Collins, 1999).

Reading scientific texts is vital for conducting research. Due to its collective nature, science requires constant literature reviews that allow us to get in touch with what others have done in a particular line of research, from which novel questions will be derived. This process also contributes to the integration of the new piece of research into the existing body of knowledge (Viniegra, 2002). However, it might be worth mentioning the reading and understanding technical materials from a line of research does not guarantee the derivation of pertinent questions, as can be observed when participants perform well on a test

1 A preliminary version of the work was published in Padilla, M.A., Tamayo, J., &

González M.L. (2010). Análisis de la posible relación entre la elaboración de preguntas informales y de investigación. IPyE: Psicología y Educación, 4, 7, 1-21. Permission granted.

M. A. Padilla Vargas, J. Tamayo and M. L. González-Torres 2

used to evaluate mastery of a particular domain and later fail to derive novel and pertinent research questions from the materials they read (Padilla, Suro, & Tamayo, 2010).

Despite its importance, the experience in research training facilities within university contexts indicate that reading technical materials (Canales, Morales, Arroyo, Pichardo & Pacheco, 2005; Pacheco, Ramírez, Palestina & Salazar, 2007; Pacheco & Villa, 2005) and the derivation of novel research questions from them are very problematic for novice researchers (Padilla, Solórzano & Pacheco, 2009).

Because it is important to find strategies that permit novice researchers to derive questions from a body of knowledge, we searched for the variables related to improvements in reading and writing. We found that specifying an achievement criterion to which the participants had to adapt in order to solve a particular task promoted effective performances (Tamayo, Padilla, & González-Torres, 2009), so we decided to explore whether manipulating this variable affected the production of research questions derived from previously read scientific materials.

Achievement criteria in educational contexts imply a set of behavioral requisites specified for a concrete situation, to which an apprentice must adapt in order to adequately solve a particular task. Achievement criteria may be specified by the task itself, by the teacher, or may be implicit.

Ibáñez (1999) investigated the correspondence between being able to identify the criteria in a text and being competent in these criteria with a sample of aspiring undergraduate students. Prior to a task, they were told the type of activity they were about to do and the criterion they were supposed to meet. After that, they were exposed to a criteria identification task in which they were asked single out the texts that provided the information that was necessary and sufficient to solve the tasks. He observed that the participants that were able to correctly identify a greater number of criteria also obtained higher scores in the production tests.

Based on these results, Mateos and Flores (2008) claimed that, although specifying and identifying achievement criteria were important, so was studying the way these criteria were specified, and set out to analyze the effects of exposing participants to different degrees of information regarding the task to be solved. They evaluated the

Evaluating of the Relationship … 3

relationship between three degrees of criterion specification (specific, non-specific or absent) and the performance in identification and production tasks, and found better levels of performance in both tasks when participants were exposed to a specific criterion.

Assuming that a taxonomy of achievement criteria depending on the type of information conveyed was needed, Ortiz, González and Rosas (2007) proposed a classification of instructions-descriptions that may be given to a participant during an experimental task. This classification was based on the amount and specificity of information that may be given before, during and/or after solving a particular task, and may refer to three elements: the stimulus conditions, the response conditions and the consequence conditions. The instructions that include response conditions also specify the achievement criterion.

Ortiz et al.’s classification divides achievement criteria into clearly defined and ordered categories, and includes, among others, the following categories:

Pertinent and Specific Description (PS). A description that contains all the elements of a contingency, and they correspond to the elements that comprise the situation in hand.

Pertinent and Generic Description (PG). The description contains only one or two of the elements of a contingency, and they correspond to the elements that comprise the situation in hand.

Absent Description (A). The description does not mention any of the elements (either relevant or irrelevant) of the situation in hand.

Because the specification of achievement criteria and the type of information these criteria convey are considered relevant for a participant to perform a task effectively, we wondered whether these aspects also were relevant when participants were required to pose research questions. In this sense, Ribes (1993) argued that the language games of the scientific practice correspond to the game of posing pertinent questions, which entails 1) posing research questions that contain relationships between facts or events and their quantitative and/or qualitative properties, and 2) establishing the categories that define the pertinence of those relationships within a disciplinary framework.

We conducted an exploratory study to evaluate whether the distinct information conveyed by the different achievement criteria affected the production of pertinent research questions, assuming that an effective


behavioral adjustment might depend on the identification and specification of these criteria.

STUDY 1

Method

Participants Twenty-four high school students, 20 females and four males, were

randomly assigned to one of three groups (n=8). This population was selected in order to reduce the probability that they were familiar with the particular theoretical domain or with psychology in general.

Design

Table 1. Experimental design

Conditions

Groups Exposure and Evaluation 1 Exposure and Evaluation 2 Evaluation

3 Group 1 N = 8 1) Read

area “x” or “y” 2) Eval.

1) Read paper “x” or “y” 2) Ev. of paper

Gen.

Specific criteria

1) Read paper “y” or “x” 2) Ev. of paper

Gen. Identification assessment

Group 2 N = 8

General criteria

Group 3 N = 8

Minimal criteria

Note: Read area “x” or “y” = read a document about the assumptions of an area (animal or human behavior); Eval. = domain evaluation of the assumptions read; Read paper. “x” or “y” = read a paper on animal or human behavior; Ev. of paper. = domain evaluation of the paper read; Gen. = generate a research question; Identification assessment = identification of the elements of a research paper. The criteria used varied in the type of information they specified: 1)

a specific criterion gave a detailed description of the task the participant had to perform (in this case, posing a research question derived from the line of research they had previously read, which had to include the


dependent and independent variables explicitly; 2) a general criterion only specified that the question had to be derived from the line of research they had previously read; and 3) the minimal criterion only specified that they had to pose a question derived from the line of research they had previously read, without further details.

Setting

The study was carried out in the high school classrooms where the students had classes. Students had their own working area and performed the task individually.

Materials

Conceptual framework of the research area (basic research on animal behavior or applied behavior analysis with humans), which contained the conceptual assumptions of each area (Skinnerian behaviorism. The order of exposure to these areas was counterbalanced within groups); 2) a glossary of technical terms of each area; 3) a reading comprehension test about the conceptual frameworks with items that required not only repetition but also elaboration of the material; 4) instructions with the corresponding criterion depending on the group; 5) two research papers, one on animal behavior and another on human behavior. The paper on animal behavior compared the number of responses emitted by four rats under continuous or intermittent reinforcement. The paper on human behavior analyzed the effects of reinforcing child social behavior while children interacted with their peers at school. Although both papers contained technical terms, we looked for papers whose style was as simple and clear as possible. They were three pages long and did not contain any terms that were not in the glossary or in the conceptual framework; 6) a reading comprehension test about the papers, with open-ended questions; 7) instructions for the identification task. In this task, participants were required to underline the following elements of one of the research papers: main terms and concepts, reported evidence, research justification and objective, research question, observed and manipulated variables, type of subjects, procedure, data, and conclusions; 8) in addition, there were pencils, erasers, pencil sharpeners, and blank sheets of paper.


Procedure Participants were individually handed the conceptual framework and

the glossary for the first area (human or animal behavior). After they finished reading the material, they were given the reading comprehension test about the conceptual framework material, which was taken away. When the participant finished the test, the experimenter immediately scored it.

A passing grade corresponded to 90% correct answers. If this criterion was not met, they were given feedback about their mistakes and were asked to correct them using the material they read. Students could not pass on to the following condition unless this criterion was met.

After that, they were given the instructions to the first test with a minimal criterion, which said they had to pose a research question. In addition, they were handed the first paper of the area corresponding to the conceptual framework they had read. After they finished reading the paper, they were given a reading comprehension test about the paper, with open-ended questions.

Again, 90% of correct answers were needed to pass on to the next condition, and they were given corrective feedback. Afterwards, they were required to pose a research question related to the paper they read. Having finished this task, participants were allowed to take a five-minute break.

Before the second test, they were given the instructions that corresponded to the group to which they were assigned. At the same time, they were handed the second paper, which was directly derived from the first. After they finished reading the paper, they were given a reading comprehension test about the paper, with open-ended questions. Again, 90% of correct answers were needed to pass on to the next condition, and they were given corrective feedback.

Afterwards, they were required to pose a research question related to the paper they read. Finally, they were exposed to the identification task, in which they were asked to underline the elements of the last paper they read. This study was conducted in a single session, in order to prevent participants from talking amongst themselves about the activities and possibly biasing the results.


Results2 Each research question posed in tests 1 and 2 was evaluated

according to the classification system established by Padilla, Solórzano and Pacheco (2009), which include three categories. The first includes proposals to manipulate the same variables present in the papers, changing only their values. The second category can be divided into three subcategories: a) proposals to evaluate correlations between variables not previously found in the literature they reviewed; b) proposals to manipulate variables not previously found in the literature they reviewed; and c) proposals to apply the theory they review to solve other theoretical or technological problems. The third category includes questions in which an experimental design is proposed that permits the validation of the theory they reviewed. This classification system implies an increasing level of complexity, where the first type of questions are simpler and third type are more complex. In addition, the number of elements correctly identified in the research papers was computed.

Figure 1 shows a comparison between the types of question posed in the first and second tests. Only four participants (1, 7, 14, and 19) showed any improvement from the first to the second test. However, the questions posed in the second test were less complex. Only one participant (19) posed a question at a level of complexity 2c in the second test. The other participants either did not show an improvement or did not pose any questions in neither test. There seemed to be no effect of the achievement criterion, since performance was poor regardless of the achievement criterion to which they were exposed.

Furthermore, there was no correlation between posing pertinent questions and being able to correctly identify the elements that comprise a research paper. Although most participants were able to correctly identify the elements that comprised the research paper they read, they still were not able to pose a pertinent question.

2 Participants’ answers were analyzed according to the criteria established in the system of

conceptual and thematic content analysis (Raigada, 2002). In each answer, the explicit and implicit terms were identified. The analysis was conducted by two independent judges, with a mean reliability of 98.22% (%Reliability = 1 – [(n1-n2) / (n1+n2)]*100).


Figure 1. Comparison between type of questions generated in Evaluations 1 and 2 by the participants in the three experimental groups.

Discussion

Several factors may account for these results. First, the population

might not be adequate, because participants anecdotally reported the low reading levels when asked about it. Additionally, results of the reading comprehension test were generally poor. All participants had to correct their answers at least once for one or more questions, which is suggestive of poor reading comprehension. Pacheco et al. (2007) suggested that reading comprehension might be a basic competence needed to effectively derive research questions, because it allows participants to establish contact with the referents presented in the text, from which they are supposed to derive their research questions. Second, materials may have been too difficult for this type of population, since technical terms from experimental psychology, with which they were not familiar, were present in the texts.

In other words, high school students were not able to pose research questions derived from a specific domain, and we wondered whether this difficulty was limited to research questions or was more pervasive and extended to a difficulty to pose pertinent questions in informal or non-academic questions. Posing pertinent questions in non-academic settings might be a precursor skill, and Pacheco et al. (2007) argued that using referents within the ordinary language domain is vital for the study of


reading and writing skills, since the contact with such referents is facilitated.

With that in mind, a second study was carried out, in which we analyzed the type of questions posed by the participants of the previous study in non-academic settings, in order to identify a possible relationship between posing informal questions and posing research questions. We found necessary to assess whether these high school students were not able to pose research questions because they were unable to pose pertinent questions in general.

STUDY 2

Method

Participants Twenty-two of the twenty-four students from the previous

experiments (several months had passed between studies, so it was not possible to locate two participants).

Setting

Same as the previous study. In this case, the tasks also were performed individually.

Materials

Participants were given a written paper with four hypothetical situations, from which they were requested to obtain a specific type of information. These situations took place in an airport with a character that was visiting a foreign destination for the first time. The information they needed to obtain included the baggage claim site (situation 1), a particular immigration proceeding (situation 2), the location of a hotel and the more appropriate means of transportation to get there (situation 3), and a place to exchange currency (situation 4). The participants were asked to imagine themselves in those situations and to pose a question that was pertinent to each of them. They were told that the question had to allow them to get all the information needed to solve the situation adequately. In addition, there were pencils, erasers, pencil sharpeners, and blank sheets of paper.


Procedure The activity took place at the end of a class and was conducted by

the teacher in charge of that class.

Results3 As can be seen in Table 2, most participants scored low when asked

to pose the questions pertinent to the four hypothetical situations. The scores used to grade the questions were assigned according to the following criteria: 0 when there was no question, 1 when the question was neither grammatical nor pertinent, 2 when the question was grammatical but not pertinent, and 3 when the question was both grammatical and pertinent. Only some participants were able to pose grammatical and pertinent non-academic questions in all four hypothetical situations (P17 and 18), or in most of them (P4, 6, and 12).

Table 2. Score obtain by participants in each one of the four

hypothetical cases

Participants Type of question generated Case 1 Case 2 Case 3 Case 4

1 1 0 1 0 2 0 0 0 0 3 0 0 0 0 4 3 2 3 3 5 0 0 0 0 6 3 2 3 3 7 0 0 0 0 8 0 0 0 0 9 0 0 0 0 10 0 0 0 0 11 0 0 0 0 12 3 2 3 3 13 0 0 0 0




Participants Type of question generated Case 1 Case 2 Case 3 Case 4

14 3 2 3 0 15 0 0 0 0 16 2 2 2 3 17 3 3 3 3 18 3 3 3 3 19 3 2 0 0 20 0 0 0 0 21 0 0 0 0 22 0 0 0 0

To assess the relationship between non-academic and research

questions, one of the four situations was randomly selected (situation 3) and the results with this situation were correlated with the results of the second evaluation of the first study, yielding a correlation index of r = 0.385. That is, there was not a strong correlation between non-academic and research questions.

Discussion The results from both studies, taken together, suggest that specifying

the achievement criterion may not be sufficient to make high school students pose novel, pertinent research questions. In addition, these same participants were unable to non-academic questions in an informal setting.

What could account for these findings? We may put forward a couple of hypotheses. Pacheco et al. (2007) have argued that the referents4 to which participants are exposed may affect their reading and writing skills, and that referents belonging to the ordinary language may facilitate performance.

With that in mind, the absence of correlation between the production of non-academic and research questions might be related to the lack of familiarity with the hypothetical situations we created, which reduced our participants’ capacity to pose pertinent questions. So, it is necessary

4 Referent refers to objects, people, events, situations, etc., about which one reads, writes

or talks.


to evaluate and control, as much as possible, our reader-writer’s history of contact with the referents to which they are exposed.

The history of contact with the referents is an element in a set of factors that intervene or modulate reading and writing interactions (others being the achievement criteria, dispositional characteristics, skills and competences, etc). This history is divided into the history of referentiality and effective situational history, the latter referring to the number of direct and indirect contacts with the referent. Both factors are dispositional elements that regulate the characteristics of the interactions that may occur between the subject and the events (Pacheco et al. 2007). The characteristics of the referent, in turn, are related to the nature and the domain of the event about which one is talking, reading or writing. Thus, the domain may be related to the ordinary language, technical language, artistic language, etc.

However, using referents that belong to the ordinary language may not ensure an effective interaction with them, since such interaction is affected by the substitutive contacts the reader-writer had in their history of referentiality with the concrete event to which they are exposed at a given time. That might explain the low scores observed in the second study. Given that participants did not have direct experience with the situation and its referents, their performances could not meet the achievement criterion, that is, they could not pose pertinent questions.

In an attempt to investigate whether their poor performance when posing questions about the four hypothetical situations was due to their lack of familiarity with the situations, we replicated the study with students from the same population (not reported here due to lack of space). We used a new hypothetical situation with elements we deemed more familiar to the participants (“Imagine you are fond of a boy or girl in your classroom and want to get to know him or her better. Write down what you’d ask them to find out: a) what kind of music they like, b) whether they want to be your friend, and c) whether they want to go out with you some time). Results were generally similar to those reported in the second study: most were unable to pose grammatical and pertinent questions.

If we add to that the requirement to read technical materials and write down research questions based on them, results from the first study are understandable, because the materials used referred to animal and human behavior. Although we looked for papers whose style was as


simple and clear as possible, the inclusion of technical terms may have made them too abstract and complex for the participants. That is, because the participants had not been exposed to similar referents throughout their history, coupled with the novelty of the situation and the technical elements of the materials, it is not surprising that they could not adequately interact with the reading material and meet the criterion we established for posing derived research questions.

It seems, therefore, that both the population was not suitable for the study because of their low levels of reading and writing and the materials we used may have been too complex for these participants. So, both the history of contact with the referent and the type of referent might hinder an effective adjustment to the task. These elements should be considered and possibly controlled in future studies.

The need to consider the history of contact and the type of referent is based on the assumption that the act of posing research questions is related to problems of reading and writing adjustments. Thus, future studies designed to analyze the production of research questions may have to consider reading and writing adjustments as basic competences. Only when a reader understands the material to which they were exposed, will they be able to write down pertinent research questions.

So, the first step will have to be to ensure that participants make effective contact with the critical referents of the texts to which they are exposed. In the case of research reports, these elements may be the research question and its justification, the dependent and independent variables, as well as the evidence that gives support to it. Although a reader has to adjust to the text as a whole, a systematic analysis has to start from the consideration and evaluation of these separate elements.

REFERENCES

Canales, C., Morales, G., Arroyo, R., Pichardo, A., & Pacheco, V. (2005). Análisis Interconductual del ajuste lector en el ámbito educativo. En C. Carpio y J.J. Irigoyen (Eds.) Psicología y educación. Aportaciones desde la teoría de la conducta (pp. 33-50). México: UNAM.

Cassany, D. (2006). Taller de textos. Leer, escribir y comentar en el aula. España: Paidós.


De la Fuente, J., Justicia, F., Casanova, P. F., & Trianes, M.V. (2005). Perceptions about the construction of academic and professional competencies in psychologist. Electronic Journal of Research in Educational Psychology, 3,1, 3-34.

Ibáñez, C. (1999). Conducta de estudio: el papel de identificar criterios en el discurso didáctico. Acta Comportamentalia, 7, 47-66.

Keys, C. W., Hand, B., Prain, V., & Collins, S. (1999). Using the science writing heuristic as a tool for learning from laboratory investigations in secondary science. Journal of Research in Science Teaching, 36, 10, 1065-1084.

Mateos, R., & Flores, C. (2008). Efectos de evaluar el grado de explicitación del criterio de ajuste sobre el desempeño de estudiantes en tareas de identificación y elaboración. Acta Comportamentalia, 16, 1, 73-88.

Ortiz, G., González, A., & Rosas, M. (2008). Una taxonomía para el análisis de descripciones pre y post contacto con arreglos contingenciales. Acta Colombiana de Psicología. 11, 1, 45-53.

Pacheco, V., & Villa, J. C. (2005). El comportamiento del escritor y la producción de textos científicos. Revista Mexicana de Investigación Educativa, 10, 27, 1201-1224.

Pacheco, V., Ramírez, L., Palestina, L., & Salazar, M. (2007). Una aproximación al análisis funcional de la relación entre las conductas de leer y escribir en estudiantes de psicología. En J.J. Irigoyen, M. Jiménez y K. Acuña (Eds.) Enseñanza, aprendizaje y evaluación en psicología. Una aproximación a la pedagogía de las ciencias. Hermosillo: UniSon.

Padilla, M. A., Solórzano, W. G., & Pacheco, V. (2009). The effects of text analysis on drafting and justifying research questions. Electronic Journal of Research in Educational Psychology, 17, 7, 1, 77-102.

Padilla, M.A., Suro, A.L. y Tamayo, J. (2010). Efectos de la exposición diferencial a los supuestos de una teoría en la elaboración de preguntas de investigación. Revista Mexicana de Psicología, 27, 2, 247-256.

Raigada, J. L. (2002). Epistemología, metodología y técnicas del análisis de contenido. Estudios de Sociolingüística, 3, 1, 1-42.

Ribes, E. (1993). La práctica de la investigación científica y la noción de juegos de lenguaje. Acta Comportamentalia, 1, 1, 63-82.


Sánchez Puentes, R. (2004). Enseñar a investigar. Una didáctica nueva de la investigación en ciencias sociales y humanas. México: Plaza y Valdés.

Tamayo, J., Padilla, M. A., & González-Torres, M. L. (2009). Efectos de criterios de logro diferenciales en la elaboración de preguntas informales, en estudiantes de preparatoria, licenciatura y posgrado. Acta Colombiana de Psicología, 12, 1, 27-39.

Viniegra, L. (2002). Un acercamiento a la crítica. Educación y Crítica: el proceso de elaboración del conocimiento. México: Paidós.

Chapter 2

EFFECTS OF EXPOSURE TO DIFFERENT ACHIEVEMENT CRITERIA

ON THE ELABORATION OF NON-ACADEMIC QUESTIONS5

Asking pertinent and grammatical questions is one of the daily tools

we can use to obtain information, to clarify a topic, to obtain knowledge or to solve problems in general. When one faces a situation in which there is an imbalance between what is known and what exists, a scenario with anomalies, contradictions, and obstacles emerges, which leads to questioning and the posing of questions in order to obtain answers that might help clarify ideas, adjust behavior or carry out specific tasks appropriately (Graesser & Olde, 2003). In a daily basis, humans are faced with situations in which they must ask questions that are pertinent to the situations.

In an educational setting, asking questions seems to be a key element to complete a comprehensive learning process. Being able to ask good questions may promote strategies to a better understanding of texts and tasks, which may lead to better academic outcomes (King, 1991). Asking questions is arguably related to the learning process, since it implies the active participation of the student, who is not limited to receiving information passed on by the teacher, and also allows the students to 5 A preliminary version of the work was published in Tamayo, J., Padilla, M.A., &

González-Torres, M.L. (2009). Efectos de criterios de logro diferenciales en la elaboración de preguntas informales, en estudiantes de preparatoria, licenciatura y posgrado. Acta Colombiana de Psicología, 12, 1, 27-39. Permission granted.


question and identify by themselves the flaws and conceptual voids that they need to fill (Graesser & Olde, 2003).

In order to ask pertinent questions, one must attend to a large number of elements present in the situations where those questions take place. Identifying these different elements in a situation is vital for asking adequate and pertinent questions. In an academic setting, this translates into students that question the world and develop a critical attitude towards the information they receive.

The studies that have been conducted to analyze individual processes and contextual elements involved in asking questions in an academic setting have been carried out within a cognitive framework. These studies focus on the evaluation of strategies and techniques that allow students to ask adequate questions or on the identification of the effects that such strategies have on other academic activities.

For instance, Davey and McBride (1986) reported significant improvement in the understanding and recollection of read materials after training sixth-graders to ask technical questions. Also, they found that the same kind of training promoted the production of more and more pertinent questions6. In a similar vein, King (1991) found an improvement in the typical classroom tasks, such as spatial reasoning, when student were trained to ask questions classified in terms of planning, monitoring, and evaluation.

Although asking questions, or being trained to ask questions, led to an improvement in academic performance, it became apparent in these studies that students ask very few questions spontaneously. Ninety-four percent of the questions are asked by the teacher. When a student asks questions, they usually lack depth, analysis, and inference (Graesser & McMahen, 1993).

Tutorial programs have been shown to improve the quality and increase the number of questions asked by 18% (Graesser, Person & Huber, cited by Graesser & McMahen, 1993). Nevertheless, the questions asked in these programs are usually based on literal information that is explicitly present in the texts, and not on the elaboration of inferences, relationships, and syntheses. It has been argued that these deficiencies occur because students fail to understand technical material. These failures consist of problems in identifying the critical 6 These authors do not make a clear theoretical or methodological distinction between

making a question and training to make a question.

Effects of Exposure to Different Achievement Criteria … 19

elements that are relevant and not easily abstracted from the texts (Graesser & McMahen, 1993).

However, teaching strategies also arguably affect the number and quality of questions asked. In some cases, a teacher may allow students to question the contents of what they are learning, whereas in other cases a teacher may organize a class in such a way that questions are reduced to a minimum, and the student becomes a mere passive agent (Graesser & Olde, 2003). Thus, an educational context that either allows or not the posing of questions may differentially affect this activity (King & Rosenshine, 1993).

In a similar vein, Freire (1985) claimed that education has been structured around what he called “the pedagogy of the response”. In this model, the teacher is conceived as the center of knowledge, leading to a mechanical process of passing on information, and students are passive agents of the educational system, that is, they do not question that knowledge; they only receive and retain the information passed on by the teacher. In this sense, education is a mechanical process that deforms students.

Freire (1985) proposed the abandonment of the “pedagogy of the response” in favor of its counterpart, the “pedagogy of the question”. In this model, the teacher is a facilitator in the development of active and critical individuals. Under this assumption, education is not reduced to the reproduction of mechanical methods centered on the teacher; it is based on the interaction between teachers and students, on questioning, and on critical thinking, which may promote social change in the short and long term.

As a consequence, asking questions in different contexts is arguably a topic that requires further analysis. However, research should not center on the process of asking questions per se, but also on the type of questions asked. Graesser and Olde (2003) found that student that scored higher on comprehensions tests ask fewer questions, but these were more concrete and pertinent, which permitted obtaining key pieces of information. Conversely, students that scored lower asked more questions, but they were less pertinent, deep, and specific, lacking the critical elements of the situation.


It seems then that some dispositional7 factors differentially affect the production of questions and their functionality in different contexts. One of such elements may be the achievement criterion. Effective performance is indicative of an adjustment to a criterion required to solve a particular task, that is, adjustment to a criterion is evidenced in an apprentice’s effective performance.

Given that specification of the achievement criteria seems to affect performance in a number of tasks, it may also affect the production of questions, both in academic and non-academic contexts. One may suppose the more clearly specified is the achievement criterion, the more pertinent to the situations the questions asked will be. Thus, in the present study, we attempted to evaluate the effect of the specificity of the achievement criterion on the production of non-academic questions. Non-academic questions are questions related to daily life, for which no specific training in a particular area of knowledge is required.

Asking non-academic questions may be an important antecedent8 to the production of scientific and research questions. In the previous chapter, we found that high school students were unable to ask pertinent research questions even after being exposed to training in a particular theoretical domain. This result led to the question of whether asking non-academic questions was a pre-requisite skill necessary for asking pertinent research questions, which meant moving back the starting point for the research on this phenomenon, and conducting a parametric evaluation of the production of questions in general. So, we exposed those same participants to a task of asking pertinent questions that were sufficient to obtain some piece of information required to solve a daily task. Again, they were unable to succeed in the task (Padilla, Tamayo & González-Torres, 2007). These results suggested we needed to explore the task of asking non-academic questions before studying the process of asking research questions.

7 Dispositional factors are the set of events that affect an interaction so as to facilitate it or

hinder it, that is, that make it more or less likely (Ribes & López, 1985). 8 Considering the absence of experimental evidence in this sense, we cannot argue that

being able to ask non academic questions is a pre-requisite to making research questions. Neither can we say that the latter types of questions are caused or automatically derived from the former type. We can say, however, that making non academic questions precedes the production of research questions from a temporal standpoint. More empirical evidence would be necessary to make causal claims.


As a consequence, we designed an experimental protocol to identify the effect of providing differential information regarding the achievement criteria on the process of asking non academic questions. We decided to compare the performances of high-school, undergraduate, and graduate students, because higher-level students usually ask more questions (Graesser & McMahen, 1993; Graesser, Person & Huber, c.p. Graesser & McMahen, 1993).

The main interest of this experiment was to analyze individual performances. To do so, we designed an experimental protocol that permitted to identify the effects of our main variable, the achievement criteria, on the performance of individual participants. This type of analysis is more compatible with the assumptions of interbehavioral psychology, whose main subject is the interaction between an individual and the objects and properties of their environment (Kantor, 1924-1926; Ribes & López, 1985).

METHOD9

Participants Twenty high-school students, 20 undergraduates majoring in

Psychology, and 20 graduate students from a Psychology program took part in this study. High school and undergraduate students received extra credit for taking part in the study. Graduate students did not receive any form of compensation for taking part in the study.

Design Participants were assigned to four different experimental groups

(n=5), exposed to two phases (see Table 3). During the first phase, they were exposed to a hypothetical situation and a minimal achievement criterion. This was used as a pre-assessment (or Baseline) in order to identify the types of questions participants asked before introducing the

9 The studies described in chapter 1 and in this chapter received a grant from CONACYT,

under protocol number 46262-H.


independent variable. During the second phase, each group was exposed to a different achievement criterion and to a second hypothetical situation. Participants were required to write down as many questions as were necessary and pertinent to the second hypothetical situation, according to the achievement criterion to which they had been exposed. Questions asked during the first and second phase were compared.


Groups

Pre-assessment Baseline Phase 1 Hypotetical case 1

Evaluation with specificity of criteria Phase 2 Hypotetical case 2

G1 N = 5

Minimal criteria of adjustment

Minimal criteria of adjustment (MC)

G2 N = 5 General criteria of adjustment (GC)

G3 N = 5 Specific criteria of adjustment (SC)

G4 N = 5

Specific criteria of adjustment with example (SCE)

Setting High-school and undergraduate students were exposed to the task in

groups within classrooms, but they solved it individually. Graduate students were exposed to the task individually in separate cubicles.

Materials Participants were provided with pencils, erasers, blank sheets of

paper, and sheets of paper with the hypothetical situations used in this protocol. The first hypothetical situation was presented along with a minimal achievement criterion. In this situation, participants read the following story: “Imagine that a new teacher is going to give you classes. You missed the first day of classes when the syllabus for the semester


was presented and explained, so you need to ask a few questions about it”.

In the second phase, all participants were presented with a different hypothetical situation and the achievement criterion that corresponded to the group to which they belonged. The group with minimal achievement criterion was exposed to the following story: “Imagine you’re at an appointment with a physician, because you’ve been feeling ill all week. He orders a blood test and prescribes some medication. You’re interested in knowing your case better and ask him some questions. Your task is to ask the corresponding questions”.

For the groups exposed to the other achievement criterion, the story was the same except for the last sentence. The general achievement group (GM) was told: “Your task is to ask questions about the diagnostics (situation 1), the tests you have to undergo (situation 2), and the medication that the doctor prescribed (situation 3). Questions should be written in a coherent manner”.

The specific achievement criterion group was told: “Your task is to ask the corresponding questions. Questions should contain all the necessary elements and be written in a coherent manner. Write down the questions you would ask about the diagnostics (a blank space was provided so that participants could write down the questions). Write down the questions you would ask about the tests you have to undergo (a blank space was provided so that participants could write down the questions). Write down the questions you would ask about the medication the doctor prescribed (a blank space was provided so that participants could write down the questions)”.

The specific achievement criterion group with example was given the same instructions as the specific achievement criterion group, but an example of a question asked appropriately was given. The example was not related to the domain of the second hypothetical situation.

Procedure Each participant sat at an individual desk (high school and

undergraduate students) or in a separate cubicle (graduate students). We thanked them for their participation and explained that the task was supposed to be solved individually. Then each participant was handed a


sheet of paper with the first hypothetical situation and was asked to call the experimenter when they had finished the task. At this moment, the experimenter grabbed the sheet with the first situation and handed the sheet with the second situation and the achievement criterion that corresponded to the group.

When participants finished solving the second task, the experimenter took the sheet of paper and thanked them again for their participation.

Results10 In both phases of the study, the number, pertinence, sufficiency, and

structure of the questions asked by the participants were given a score. Appendix A shows some examples of the questions asked by the participants and their respective scores. The number of questions was defined as the total number of questions asked in each situation. The pertinence was defined as the correspondence between the question and the situation under which the question was asked. Questions were given a score of 1 when they corresponded to the situation and were necessary to obtain the required information, and a score of 0 otherwise.

The sufficiency was defined in terms of whether the whole set of questions asked by the participants permitted to obtain all the necessary information in each case. The whole set of questions was given a score of 1 when it was deemed sufficient to obtain all the information needed, and a score of 0 otherwise.

Finally, the structure refers to whether questions were grammatical. We assigned them different scores based on the presence (or absence) of syntactical elements needed to properly ask a question, such as the question mark or spelling errors, among others. Questions were given a score of 2 when they were grammatical, 1 when there was some mistake, and 0 when questions were not structured.

Within-subject and between-group comparisons of the questions asked in both phases were conducted. Figure 2 shows the number of




questions asked by the participants belonging to the three populations. There appears to be an effect of the specificity of the achievement criteria in all three populations. Both groups with specific criteria showed an increase in the number of questions asked when the two phases are compared, whereas groups with minimal or general criteria did not show a marked difference between phases. The differences among groups, however, did not reach statistical significance. Differences between the populations, on the other hand, were significant, both the in first (F = 4.074, p = 0.022) and the second phase (F = 6.616, p = 0.003).

Figure 2. Comparison between the number of questions generated in phases 1 and 2 by the participants of the three populations evaluated. Left: high school; middle: college; right: postgraduate. It also shows the type of criteria that were exposed to the different groups. MC: Minimal Criteria; GC: General Criteria; SC: Specific Criteria; SCE: Cpecific Criteria with Example.

Data for pertinence were more variable. For high-school students, there seems to be an effect of the specificity of the achievement criteria, because questions asked by groups exposed to general, specific and specific with example achievement criteria were more pertinent in the second phase, whereas the group exposed to minimal criterion shows no difference between phases.

For undergraduate students, minimal and general achievement criteria brought a slight increase in the pertinence of questions in the second phase, whereas groups exposed to specific and specific with examples showed either no difference between phases or a slight decrease. For graduate students, there was no difference between phases or among groups, possibly due to a ceiling effect (see Figure 3). Differences that might have been related to the achievement criteria,


however, were not statistically significant. Again, differences between the populations, on the other hand, were significant, both the in first (F = 8.186, p = 0.01) and the second phase (F = 3.53, p = 0.036). Graduate students asked more pertinent questions than undergraduates, who in turn asked more pertinent questions than high-school students.

Figure 3. Comparison between the percentage of questions generated by group and phase. Left: high school; middle: college; right: posgradute. MC: Minimal Criteria; GC: General Criteria; SC: Specific Criteria; SCE: Specific Criteria with Example.

Figure 4. Comparisons between the number of subjects per group generated sufficient questions in relation to the hypothetical case. Phase 1: hypothetical case related to class rules; phase 2 was divided in hypothetical case 1, related to medical diagnostic; hypothetical case 2, related to exams and medical test; and hypothetical case 3, related to medication use. Left: high school; middle: college; rigth: posgraduate. MC: Minimal Criteria; GC: General Criteria; SC: Specific Criteria; SCE: Specific Criteria with Example.


Figure 4 shows the number of participants that asked sufficient questions. For high-school students, there seemed to be an effect of the achievement criteria when phases 1 and 2 are compared. The effect seems clearer for the group exposed to a specific criterion with example, whose participants asked more sufficient questions for all three cases in the second phase. Lower scores were observed for the second case.

For undergraduate students, there also seemed to be an effect of the achievement criteria on the sufficiency of the questions asked when phases 1 and 2 are compared: a more specific achievement criterion was related to more sufficient questions. For graduate students, the number of participants that asked sufficient questions was lower with minimal and general criteria. For the group exposed to the general criterion, the number of participants that asked sufficient questions was low in the first phase and variable in the second, whereas this measure declined from phase 1 to phase 2 for the group exposed to a minimal criterion. Again, differences that might have been related to the achievement criteria, however, were not statistically significant.

A comparison between the different populations showed that levels of sufficiency for undergraduate and graduate students were higher in the first phase, whereas most participants could ask sufficient questions regardless of the population to which they belonged in the second phase, especially related to the first case. With the other cases, undergraduates and graduate students scored higher than high-school students, especially those from the groups exposed to specific criteria. These differences did not reach statistical significance.

Finally, Figure 5 shows the number of structural errors made. High-school students made more structural mistakes (spelling errors, missing question marks, etc.), followed by unstructured questions (sentences not phrased as a question) and correct questions to a lesser degree. Undergraduate and graduate students asked more questions with the correct structure, followed by questions with an incorrect structure and unstructured questions to a lesser degree. No differences were observed regarding the specificity of the achievement criterion. A statistical analysis showed that high-school students asked more incorrect or unstructured questions in the first phase, but no in the second phase, which suggests that the mere exposure to the task regardless of the achievement criteria made the populations more homogeneous as far as the structure of the questions is concerned.


Figure 5. Percentage and type of errors made by the participants in phases 1(P1) and 2 (P2). Left: high school; middle: college; rigth: posgraduate. MC: Minimal Criteria; GC: General Criteria; SC: Specific Criteria; SCE: Specific Criteria with Example.

DISCUSSION

In this study, we investigated the effect of providing participants

with differential information regarding the achievement criterion in the task of asking non-academic questions. Although there were some differences among groups regarding the variables we evaluated (number of questions, pertinence, sufficiency, and structure), the differences that might have been attributed to the achievement criteria were not significant.

In the first phase, participants were given minimal instructions as to how they should ask the questions. In the second phase, different groups received different instructions that varied according the specificity of the information that was provided. However, participants given more specific information did not perform any better than participants given less specific information. These results differ from those obtained by Ibáñez (1999) and Mateos and Flores (2008), who found that differential achievement criteria and the type of information conveyed in them affected the way their participant solved a task.

On the other hand, the scores for the number of questions asked, pertinence, and structure of the questions asked by graduate students were higher than the scores for the other populations. These differences


were significant in both the first and the second phase, except for the score for structure, which did not differ among groups in the second phase. This result suggests that the mere exposure to a criterion, regardless of the amount of information conveyed in it, made the scores more homogeneous among populations in the second phase.

Some tentative hypotheses may be put forward to explain this last finding. First, academic experience seems to improve the ability to structure and ask questions. Graesser and McMahen (1993) and Graesser, Person and Huber, (as cited in Graesser & McMahen, 1993) reported that students at higher levels of education, especially those under a tutorial program, usually ask more questions than those in high school. In addition, the quality of the questions asked also was superior, although they revolved around the materials students read, and did not transcend to other situations analyzed during tutoring sessions. So, graduate programs, which usually entail a more direct relationship between student and advisor, may promote inquiry, skepticism, and critical reasoning toward the materials to which students are exposed.

Freire (1985) argued that education centers on the answer, while ideally it should revolve around the question. Our results seem to indicate that high-school educational programs and didactic strategies are based on the pedagogy of the answer, which may prevent students from making pertinent, sufficient, and structured questions. This suggests that a mechanical model of knowledge transmission centered on the teacher is predominant at lower levels of education (Graesser & Olde, 2003). It seems that the quality of education has not received much attention when the public policies extended educational coverage. The goal of extending educational coverage caused classrooms to be filled with students that need to be guided by a single teacher.

This situation makes it difficult for teachers to conduct an individualized follow-up with their students and compels them to organize their didactic strategies so as to cover as much of the syllabus as possible in little time. Conversely, since graduate programs generally consist of a reduced number of students, an individualized follow-up is more workable. Moreover, because a higher level of mastery of some specialized topic is required, teaching methods center on the more ideal pedagogy of the question (Freire, 1985).

Other elements might have contributed to these results. First, higher levels of experience and/or familiarity with the different hypothetical


situations may have promoted higher scores. Second, results from high school students may have been affected by their level of understanding of the instructions, since previous studies have shown that reading comprehension in this particular population is poor (Morales, Pichardo, Arroyo, Canales, Silva & Carpio, 2005). Finally, the type of task we used might have affected the way participants asked questions. The materials we used and the lack of familiarity with the technical terms in them, the distinct hypothetical scenarios and the lack of familiarity with these scenarios, among other factors, might have affected the type and number of questions asked, as well as their pertinence and sufficiency. Further studies with different experimental protocols might be necessary to assess these possible biases.

So, the effects we observed in this study were more likely due to characteristics of the population than the variable we manipulated (achievement criterion). Thus, it may be worthwhile to carry out more research controlling for participants’ experience/familiarity with the hypothetical situations to which they are exposed, as well as to conduct longitudinal investigations on the development of behavior of asking questions from childhood through adulthood.

Likewise, it also may be worthwhile to assess the efficiency of the various procedures designed to train the behavior of asking these types of questions in more controlled situations (Davey & McBride, 1986; Graesser & Olde, 2003; King, 1991). Finally, these results point to deficiencies in the current teaching methods used in high school and colleges, and it may be necessary to reevaluate the pedagogical strategies employed in these settings.

REFERENCES

Davey, B., & McBride, S. (1986). Effects of questions-generation training on reading comprehension. Journal of Educational Psychology, 78, 4, 256-262.

Freire, P. (1985). Por uma pedagogia da pergunta. Rio de Janeiro: Paz e Terra.

Graesser, A. C., & McMahen, C. L. (1993). Anomalous information triggers questions when adults solve quantitative problems and


comprehend stories. Journal of Educational Psychology, 85, 136-151.

Graesser A. C., & Olde, B. (2003). How does one know whether a person understands a device? The quality of the questions the person ask when the devise breaks down. Journal of Educational Psychology. 95, 3, 524-536.

Ibáñez, C. (1999). Conducta de estudio: el papel de identificar criterios en el discurso didáctico. Acta Comportamentalia, 7, 47-66.

Kantor, J. R. (1924-1926). Principles of psychology. New York: Alfred Knopf.

King, A. (1991). Effects of training in strategic questioning on children’s problem-solving performance. Journal of Educational Psychology, 83, 3, 307-317.

King, A., & Rosenshine, B. (1993). Effects of guided cooperative questioning on children´s knowledge construction. Journal of Educational Psychology, 61, 2, 127-148.

Mateos, R., & Flores, C. (2008). Efectos de evaluar el grado de explicitación del criterio de ajuste sobre el desempeño de estudiantes en tareas de identificación y elaboración. Acta Comportamentalia, 16, 1, 73-88.

Morales, G., Pichardo, A., Arroyo, R., Canales, C., Silva, H., & Carpio, C. (2005). Enseñanza de la psicología a través de la lectura: un ejemplo de abordaje experimental a la comprensión de textos. En C. Carpio, & J.J. Irigoyen-Morales (Comp). Psicología y Educación: aportaciones desde la Teoría de la Conducta. México: UNAM Fes Iztacala.

Padilla, M. A., Tamayo, J., & González-Torres, M.L. (2007a). Efectos de la especificación del criterio de logro en la elaboración de preguntas de investigación. En S. Carvajal (Editor). 2007 Avances en la Investigación Científica del Centro Universitario de Ciencias Biológicas y Agropecuarias. México: Universidad de Guadalajara. ISBN. 978-970-27-1280-0.

Padilla, M. A., Tamayo, J., & González-Torres, M. L. (2007b). Elaboración de preguntas informales y su posible relación con la formulación de preguntas de investigación. En S. Carvajal (Editor). 2007 Avances en la Investigación Científica del Centro Universitario de Ciencias Biológicas y Agropecuarias. México: Universidad de Guadalajara. ISBN. 978-970-27-1280-0.


Raigada, J. L. (2002). Epistemología, metodología y técnicas del análisis de contenido. Estudios de Sociolingüística, 3, 1, 1-42.

Ribes, E., & López, F. (1985). Teoría de la Conducta. Un Análisis de Campo y Paramétrico. México: Trillas.

APPENDIX A Examples of questions asked by our participants and the way each

was classified. Each criterion was independent, that is, a correct but unstructured question could be either pertinent or non pertinent, and be part of a set of questions that were either sufficient or insufficient as whole.

a) Example of a question classified as pertinent to the first

hypothetical situation (class syllabus): - What type of work is required and how is it graded?

b) Example of a question classified as non pertinent to the first hypothetical situation (class syllabus):

- What are your impressions of the group? c) Example of a question classified as pertinent to the second

hypothetical situation (medical exams): - What are these tests for?

d) Example of a question classified as non pertinent to the second hypothetical situation (medical exams):

- What if he only sends me to do the test without saying anything?

e) Example of a set of questions classified as sufficient to the first hypothetical situation (class syllabus):

- What rules apply to this term? - Can I have a copy of the syllabus?

f) Example of a set of questions classified as non sufficient to the first hypothetical situation (class syllabus):

- What is the name of the new teacher? - Is it the same syllabus as the previous teacher’s? - Did he leave homework?

g) Example of a set of questions classified as sufficient to the second hypothetical situation (prescribed medicine):


- What effects will the medicine have in my body? - How long do I have to take them for? - How often do I have to take them? - Do they have side effects?

h) Example of a set of questions classified as non sufficient to the second hypothetical situation (medical exams):

- When do I have to pick them up? - What time do I have to pick them up?

i) Example of a correct structured question - How are we going to be graded?

j) Example of an incorrect structured question - What is the rules in this classroom?

k) Example of an unstructured question - I’d ask him if he believes I have a serious illness.

Chapter 3

VARIABLES THAT AFFECT THE PROCESS OF ASKING AND JUSTIFYING

RESEARCH QUESTIONS One of the main goals of those who teach science is that novice

researchers may eventually conduct research in a novel and effective way (De La Fuente, Justicia, Casanova & Trianes, 2005; Sánchez Puentes, 2004). This entails a set of writing skills, because they are part of the scientific practice (Cassany, 2006). According to Keys, Hand, Prain y Collins (1999), besides spreading knowledge and divulging empirical evidence, writing scientific texts promotes the generation of new knowledge.

To do so, scientists must read and write at all functional levels, especially at the more complex ones, which entails identifying the distinct elements that comprise a text, relate them to previously read materials, defend ideas and derive novel questions, justifying them based on the technical literature reviewed (Fuentes, 2005).

In order to analyze the relationships between reading and writing skills, Pacheco, Ramírez, Palestina and Salazar (2007) asked undergraduate psychology students to read a scientific article and to carry out a series of activities: to draft an abstract, to offer an opinion about the text, to identify its elements, to ask a research question derived from it, and to conceive a research project.

Variables That Affect the Process … 37

Most participants fared better in the tasks that involved intrasituational interactions, as opposed to activities that required a more complex level of interaction (extra or transituational).

To analyze further the variables upon which the process of asking and justifying derived, novel, and pertinent research questions depends, Padilla, Solórzano and Pacheco (2009) evaluated whether training to identify and to draft different types of paragraphs of an experimental article enabled graduate students to write down a justified research question. Eleven graduate students were assigned to an experimental and a control group. The experimental group was exposed to a pre-assessment (or Baseline) over two sessions during which they read two research papers, two training exercises during which they read two distinct research papers, and a post-assessment. The control group was exposed to the same conditions, except that they were trained neither to identify nor to elaborate on the elements of the papers (see Table 4). These participants only read the papers used for training the experimental group, because they were supposed to derive a research question from these articles (for control reasons, it was necessary that both groups were exposed to the same articles). In addition, we wanted to make sure that the act of reading these materials did not by itself cause participants to ask and justify research questions.

Data were analyzed according to the following criteria: the elements of the justification of the research question were first classified into title, definitions, evidences, derived approaches and research question, and the objective. A score of one was given if the answer was sufficient, half a point if it was insufficient, and zero if it was inadequate or absent. One other point was given if participants could identify the dependent variable and another if they could independent variable in the justification they wrote. These scores were added and expressed as a percentage of the total correct answer. Based on a classification proposed by Padilla et al. (2009), we identified the level at which the research question was asked (see Table 5).

Before training, participants had a really hard time asking and justifying their research questions, which suggests that reading and writing occurred at a less complex level (intrasituational). After training, however, their performance improved substantially, with questions asked at extrasituational levels. In addition, the experimental group performed


better than the control group, at both asking the questions and justifying them.

Table 5. Types and characteristics of the research questions

1. Variables are manipulated and evaluated and only chance the value of one variable. 2. a) evaluate relationship (between variables) not considered in the field of knowledge, b) is evaluated and manipulated variables that are relevant to the problem of interest, and c) apply the principles of scientific theory to explain a social concrete (related to technological research) or a scientific problem (related to basic research). 3. Experimental preparations are proposed to validate the facts of a theory. Regarding the task of identifying the paragraphs that comprise a

scientific paper (Exercise 1), most participants had a hard time doing it, although it involved behavior at a simpler level (intrasituational). In the second exercise, when participants had to identify the different types of questions, to recognize the experimental variables (dependent and independent), and to write down a paraphrase of the materials they read (extrasituational level), performance was generally poor, even worse than in the first exercise, given the greater complexity involved in these tasks.

Figure 6. Levels that were generated the research questions, for each participant in the pre-assessment (or Baseline) and in the sessions 1 and 2 of the Evaluation.


Regarding the justification of the research question, the experimental group performed better than the control group (Figure 6), which suggests that the training to which they were exposed raised the level at which their questions were justified. These results are consistent with the suggestion by Pacheco et al. (2007) that the level at which one reads corresponds to the level at which one writes. Apparently, exposing novice researchers to technical materials and explicitly training them to draft a scientific paper may improve the behaviors of asking and justifying research questions derived from the papers they read previously (Figure 7).

Figure 7. Percentage of corrects responses obtained by the participants in the experimental and control groups on the justification of research questions asked in the pre-assessment (or Baseline BL) and the two evaluation sessions.

In the same vein, Padilla, Fuentes and Pacheco (submitted) analyzed the effects of a corrective training for the identification and elaboration of some of the elements that comprise an empirical paper on the behavior of asking and justifying research questions. Twenty undergraduate students were assigned to an experimental or control group. The experimental design was similar to the one in the previous study, except that the experimental group was exposed to a corrective training in the identification and the drafting of empirical papers. This was done to promote performance at intrasituational and extrasituational levels, respectively. Both quantitative (Figure 8) and qualitative analyses were conducted. For the quantitative part, we recorded the percentage of correct responses in each condition, using a scale especially designed for this study (for simplicity, only the data from the pre- and post-

0

20

40

60

80

100

BL Evaluation SS1 Evaluation SS2

% o

f cor

rect

res

pons

es

Experimental

Control

Experimental conditions


assessments are presented). In addition, we also identified a) the number of words used in the proposal, under the assumption that the number of words used in the justification would be increased by training, since participants would present more arguments, b) the propriety with which the concepts were used in the pre- and the post-assessments, c) the congruence between the proposed research question and its justification based on the papers participants read, and d) the functional level at which the research questions were asked and justified.

As shown in Table 6, the corrective training to which the experimental group was exposed arguably promoted the development and justification of research questions at extrasituational levels.

Table 6. Number of words used by the experimental and control

groups to develop and justify their research question and the Propriety and Congruence of the research questions generated,

and the Functional Level at wich they were written (E= Extrasituational; I= Intrasituational; X= No-response*)

Experimental Group

Participant Condition No. of

Words Propriety Congruence Functional Level

P1 Pre 380 No Yes E Post 647 Yes Yes E P2 Pre 552 No Yes I Post 816 Yes Yes E P3 Pre 27 No Yes X Post 1002 Yes yes E P4 Pre 310 No Yes X Post 390 No Yes X P5 Pre 380 No Yes I Post 369 No Yes I P6 Pre 140 No No I Post 305 No No I P07 Pre 269 No No I Post 271 No Yes I P8 Pre 388 Yes Yes E Post 435 Yes Yes E P9 Pre 410 No Yes I Post 571 Yes Yes E P10 Pre 352 No Yes I Post 1974 Yes Yes E


Control Group

Participant Condition No. of Words Propriety Congruence Functional

Level P11 Pre 219 No Yes I Post 140 No Yes I P12 Pre 384 Yes Yes E Post 846 Yes Yes E P13 Pre 357 No No X Post 585 No No I P14 Pre 118 No Yes X Post 263 No Yes I P15 Pre 183 No No I Post 467 No Yes I P16 Pre 323 No Yes X Post 233 No Yes I P17 Pre 207 No Yes I Post 209 No Yes I P18 Pre 246 No No I Post 209 No No I P19 Pre 445 Yes Yes I Post 406 Yes Yes I P20 Pre 196 No No I Post 493 No Yes I Looking at the number of words used in the proposals, with one

exception, participants in the experimental group used substantially more word in the post-assessment, compared to the pre-assessment, as expected, which may suggest a greater number of evidences and further reasoning. This effect was not as marked in the control group, although these participants read the same papers.

Regarding the congruence between the proposed research question and its justification based on the papers participants read, two participants of the experimental group presented arguments that were not congruent with the research questions they posed, as opposed to four participants of the control group.

Figure 8. Percentage of correct responses obtained by the participants of experimental and control groups during the pre-assessment (or Baseline) and the evaluation.


As for the proper use of the technical terms when justifying the research question, we found that five participants from the experimental group started using them in a proper manner after being exposed to the training (although one of them already did that in the pre-assessment), which was related to a higher functional level (extrasituational). A similar effect was not observed in the control group, which suggests that only reading the papers without specific training in the identification and elaboration of their elements does not promote the proper use of the technical terms.

Considering these results, Padilla and Fernández (submitted) evaluated the effects of varying the textual characteristics on the experimental paper (making them unstructured) on the percent correct and the functional level (intrasituational or extrasituational) at which undergraduate students proposed and justified research questions. Twenty students from a public university were again assigned to an experimental and a control group. The experimental design was similar to that of previous studies. However, some of the elements of the materials we used were modified to make the papers unstructured: 1) the elements of the research question (e.g., the dependent variable, the independent variable, and the relationship between them) could not be clearly identified, 2) the pertinence of the research question was not clearly stated and the arguments for it did not appear in the same paragraph as the research question, and 3) the empirical evidence related to the different elements of the research questions was not fully detailed. These changes were brought about by the observation that a paper with these three characteristics resulted in consistently poor performances in a previous study (Padilla et al., 2009). Because these were the only differences we could find with the materials we used in other sessions, we decided to put to a test the hypothesis that these differences produced the results we observed, by manipulating the characteristics of the empirical papers to which the participants were exposed.

Figure 9 shows the results of this study. Compared to previous studies, when participants were exposed to structured papers, the percentage of correct answers, the functional level at which participants and developed and justified their research questions, and the propriety with which they used the technical terms were poorer.


Figure 9. Percentage of correct responses obtain by the participants in the pre-assessment (or Baseline) and Evaluation conditions.

In addition, the percentage of correct answers in identification and elaboration tasks also was poorer, despite corrective training. These results are consistent with Padilla et al.’s finding that performance was generally poorer when participants were faced with papers later classified as complex or unstructured.

Therefore, when untrained readers, such as a novice researcher with no or little experience reading experimental papers, are exposed to a text in which the research questions and its pertinence are not clearly stated, its elements and the relationship between them are not easily identified, or the evidence that supports it is not adequately presented or organized, performance in tasks of identification and elaboration is hindered (Padilla et al., 2009; Padilla & González, submitted).

Further support for these claims is being currently obtained in our laboratory. This time, we exposed six expert researchers to structured and unstructured papers and analyzed the way those participants posed and justified a research question (Padilla & González, 2013, in preparation). The experimental design was the same as the previous study. Results showed in increment in the average percentage of correct answers from the pre-assessment (61%) to the post-assessment (93%), both for those exposed to structured articles and those exposed to unstructured ones, which suggests an effect of training. It is remarkable that those participants had a relatively good performance in the pre-assessment compared to novice participants exposed to structured texts, who usually obtain about 30% of correct answers in this condition, which suggests that the experience of the expert researchers in reading and writing technical papers minimizes the effects of unstructured experimental articles. Moreover, the effect of training

0

20

40

60

80

100

P01 P02 P03 P04 P05 P06 P07 P08 P09 P10

% o

f cor

rect

resp

onse

s

Participants

Base LineEvaluation


seemed a little stronger when compared to undergraduate students reading structured texts.

Of course, since we have only run six participants, we cannot make bolder claims about the effect and its generality. So, we need to conduct this experiment with a larger sample size, and continue to analyze the phenomenon systematically, with a parametric analysis of the variables that affect the production of questions, both academic and non academic, as well as the relationship between them.

REFERENCES

Cassany, D. (2006). Taller de textos. Leer, escribir y comentar en el aula. España: Paidós.

De la Fuente, J., Justicia, F., Casanova, P. F., & Trianes, M. V. (2005). Perceptions about the construction of academic and professional competencies in psychologist. Electronic Journal of Research in Educational Psychology, 3, 1, 3-34.

Fuentes, T. (2005). Repertorios recurrentes de la comprensión lectora reconstructiva. Tesis doctoral, Universidad de Guadalajara, Guadalajara, Jalisco, México.

Keys, C. W., Hand, B., Prain, V., & Collins, S. (1999). Using the science writing heuristic as a tool for learning from laboratory investigations in secondary science. Journal of Research in Science Teaching, 36, 10, 1065-1084.

Pacheco, V., Ramírez, L., Palestina, L., & Salazar, M. (2007). Una aproximación al análisis funcional de la relación entre las conductas de leer y escribir en estudiantes de psicología. En J.J. Irigoyen, M. Jiménez , & K. Acuña (Eds.), Enseñanza, aprendizaje y evaluación. Una aproximación a la pedagogía de las ciencias (pp. 247-275). México: UniSon.

Padilla, M. A., Solórzano, W. G., & Pacheco, V. (2009). The effects of text analysis on drafting and justifying research questions. Electronic Journal of Research in Educational Psychology, 17, 7, 1, 77-102.

Padilla, M. A., Fuentes, N., y Pacheco, V. (submitted). Análisis de los efectos de un entrenamiento correctivo en la elaboración y fundamentación de preguntas de investigación. Revista Mexicana de Investigación Educativa.


Padilla, M. A., y Fernández, G. (submitted). Análisis del efecto de manipular algunas características textuales del referente en la lectoescritura de textos científicos. Revista Latinoamericana de Psicología.

Padilla, M. A., y González, J. (2013). Análisis del efecto de exponer a artículos incompletos a investigadores experimentados. Manuscript in preparation.

Sánchez Puentes, R. (2004). Enseñar a investigar. Una didáctica nueva de la investigación en ciencias sociales y humanas. México: Plaza y Valdés.

GENERAL CONCLUSION Making proper non academic questions contribute to posing research

questions? If so, what is the relationship between these two types of questions? The present book attempted to shed light on these inquiries. The evidence we gathered so far with the studies we have conducted suggest that the answer to the first question is affirmative. The behavior of asking questions seems intimately related to reading and writing skills; making a good research question implies that it should be not only novel, pertinent, useful, and viable, but also derived from the literature of the area of interest, because science is not a lonely enterprise, but highly collaborative. This means that significant breakthroughs in science are more likely with the participation of hundreds of researchers working on the same problems, from different perspectives, in different laboratories, with diverse experimental protocols and populations, with more or less clear and unified criteria and boundaries. The advancement of science is made possible by the contribution of one’s work to the task of building a solid and coherent body of knowledge.

Scientific knowledge is similar to a wall with irregular holes in it. The job of scientists is to identify what kinds of research questions allow for a pertinent experimental preparation that will provide the evidence that might fill in a specific void in the wall of science. This is the only way a significant contribution can be made. Otherwise, the work would be of no use to anyone, since it would not contribute to the advancement of a particular area and could remain an isolated piece of knowledge.

As consequence, it is imperative that reading comprehension is fostered in novice researchers, so they will be able to pose a derived research questions that is coherent with the existing body of knowledge in a particular area. In order to propose a properly justified research


question that contributes to the advancement of a particular area, it is necessary to conduct exhaustive searches into the existing literature, to adequately understand the materials read, and to identify the “holes in the wall of science”.

Of course, the research question has to be original and not a mere repetition of what has already been done, varying simple parameters. This has no academic merit, and it is a waste of time and resources. Our results show that participants often repeated the questions that had already been investigated in the papers they read, or only proposed minimal non pertinent variations to the independent variables manipulated in these materials. Additionally, participants often had a really hard time justifying the proposed research questions, despite being required to derive them from the evidence provided in the papers they read. This difficulty may stem from the fact that justifying a derived research question requires the exercise of writing skills at an extrasituational level, and our participants’ writing skills were generally poor, hardly exercised, and occurred at lower functional levels. This was very evident when they only repeated the information presented in the papers (sometimes even unrelated to the proposed question), whereas the proper way of justifying would be to restructure and elaborate on these bits of information so as to come up with arguments that were coherent with the new questions they posed. Only experienced researchers were able to do it, which suggests that academic and investigative practice improves adjustment to the expected achievement criterion.

The results reported here seem to indicate a direct relationship between educational experience and the ability to ask research questions. Performance of undergraduate and graduate students was initially poor in general, but improved substantially according to their academic status after being exposed to a specific training in reading and writing technical materials, especially if it was corrective. This training aimed at developing skills at both intrasituational (identification) and extrasituational (elaboration) levels. High school students, on the other hand, did not seem to benefit from training and were able to pose neither research questions nor non academic questions, even when the achievement criterion was clearly specified.

Conversely, more experienced researchers performed relatively well from the beginning, and achieved nearly perfect scores after training, even when the papers to which they were exposed were modified to

General Conclusion 49

become disorganized and complex. When undergraduate students were exposed to the same materials, their average scores did not exceed 50% of correct answers.

We may conclude that reading and writing skills in general, and of technical materials in particular, affect how we make both academic and non academic questions, which may be related to our referential history, that is, our lack of familiarity with these types of texts. To make things worse, our data also suggests that education is based on what Freire called “the pedagogy of the answer”, which conceives the teacher as the center of knowledge, whose job is to impart bits of information mechanically. The student, therefore, becomes a mere passive agent of the educational system, someone that does not question those bits of information, that does not make questions and that does not ponder on them. That results in students that do not know how to make questions in any form, neither non academic nor research questions.

Another noteworthy result is that students with greater educational experience (undergraduate and graduate students) are usually better at asking non academic questions, as shown in Chapter 2. The same was observed for research questions. This relationship between being able to ask pertinent research and non academic questions and one’s educational level might be due to the selection process prospective students undergo. Those who enroll in higher education might be the ones that already had better reading and writing skills, and that were more likely encouraged to express a critical attitude towards knowledge throughout their formative years, which means that questioning to themselves and posing those questions to their teacher were common practices for them.

For that reason, it is important to restructure the education system so as to promote critical reasoning at all educational levels, to create students that question what they are told, that are skeptical, that inquire their surroundings and that pose questions to their teachers. Also, it is vital to explicitly promote training of reading and writing skills from the more basic educational levels, so that students are able to read both technical and non technical materials, which in turn will provide them with the necessary precursors to make good research questions, should they choose this career path. Several diagnostic analyses have shown that undergraduate Psychology students at the Mexican National Autonomous University (UNAM) have poor reading and writing skills. However, there are no projects especially designed to improve these abilities in the


curriculum. Quite the opposite happens in Colombia, for example, where there are programs especially designed to improve research skills in which reading, textual analysis, and writing are explicitly trained (Guerrero, 2007).

Finally, there seems to be a relationship between making good non academic questions and making good research questions, which may be a starting point in the training of novice researchers. In the same vein, considering that the main objective of science is to produce knowledge, teaching to conduct research must entail, among other things, training the capacity to pose original problems based on the reconstruction of various approaches to a particular subject matter. Part of the teacher’s job, therefore, is to entice students’ curiosity, so that they are more likely to pose creative and practical problems. Another part should be to encourage the critical reading of what has been done regarding a particular topic, so that students may come up with their own problems, amenable to investigation. This might be the first step towards training critical researchers that are engaged on making high-level science that will ultimately impact on the development of a progressively less unequal and more educated society.

REFERENCE

Guerrero, M. E. (2007). Formación de habilidades para la investigación desde el pregrado. Acta Colombiana de Psicología, 10, 2, 190-192.

INDEX

A

academic performance, 18 academic settings, 8, 9 adjustment, 4, 13, 20, 22, 48 adulthood, 30 adults, 30 advancement, 47, 48 animal behavior, 5, 6 assessment, xvi, 4, 21, 22, 37, 38, 39, 41,

42, 43, 44

B

baggage, 9 basic education, 49 basic research, 5, 38 behaviorism, 5 behaviors, 39 benefits, ix, xv blood, 23

C

childhood, 30 children, 5, 31 classes, 5, 22 classification, 3, 7, 37

classroom, 12, 18, 33 colleges, 30 Colombia, 50 community, ix, xv compensation, 21 complexity, 7, 38 comprehension, 5, 8 congruence, 40, 41 construction, 14, 31, 45 content analysis, 7, 10, 24 contingency, 3 control group, xvi, 37, 38, 39, 40, 41, 42,

43 correlation(s), 7, 11 critical thinking, 19 currency, 9 curriculum, 50

D

deficiencies, 18, 30 dependent variable, 37, 43 depth, ix, xv, 18 diffusion, 1 draft, 35, 37, 39

E

education, 1, 19, 29, 49

Index 52

educational experience, 48, 49 educational programs, 29 educational system, 19, 49 educators, ix, xv elaboration, 5, 18, 39, 43, 44, 48 environment, 21 evidence, 1, 5, 13, 20, 35, 43, 44, 47, 48 exercise, 1, 38, 48 experimental design, 7, 39, 43 exposure, 5, 27, 29

F

financial, ix, xi, xv flaws, 18

G

graduate program, 29 graduate students, xvi, xvii, 21, 23, 25,

27, 28, 37, 48, 49

H

high school, xv, xvi, 4, 5, 8, 9, 11, 20, 23, 25, 26, 28, 29, 30

higher education, 49 history, 12, 13, 49 homework, 32 hotel, 9 human, 4, 5, 6, 12 human behavior, 4, 5, 12 hypothesis, 43

I

ideal, 29 identification, 2, 3, 4, 5, 6, 18, 39, 43, 44,

48 immigration, 9 improvements, xvii, 2

independent variable, 5, 13, 22, 37, 43, 48

individuals, 19 inferences, 18 integration, 1

J

justification, 5, 13, 37, 39, 40, 41

L

lead, 17 learning, 14, 17, 19, 45 learning process, 17 light, 47

M

materials, xvii, 1, 2, 8, 12, 13, 18, 29, 30, 35, 37, 38, 39, 43, 48, 49

matter, 50 medical, 26, 32, 33 medication, 23, 26 medicine, 32, 33 methodology, xvii Mexico, ix, xv music, 12

O

obstacles, 17 organize, 19, 29

P

participants, xvi, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 20, 21, 22, 23, 24, 25, 27, 28, 30, 32, 37, 38, 39, 40, 41, 42, 43, 44, 48

pedagogy, 19, 29, 49 permit, 1, 2

Index 53

poor performance, 12, 43 population, xvi, 4, 8, 12, 13, 27, 30 preparation, 46, 47 principles, 38 probability, 4 problem-solving, 31 project, 35 psychologist, 14, 45 psychology, 4, 8, 21, 31, 35 publishing, ix, xv

Q

question mark, 24, 27 questioning, 17, 19, 31, 49

R

reading, xvi, xvii, 1, 2, 5, 6, 8, 9, 11, 12, 13, 30, 35, 37, 43, 44, 47, 48, 49, 50

reading comprehension, xvi, 5, 6, 8, 30, 47

reading comprehension test, 5, 6, 8 reasoning, 18, 29, 41, 49 reconstruction, 50 reinforcement, 5 reliability, 7, 10, 24 reproduction, 19 researchers, ix, xv, xvi, xvii, 1, 2, 35, 39,

47, 48, 50 resources, 48 response, 3, 19, 40 rules, 26, 32, 33

S

school, xv, xvi, 5, 21, 22, 25, 26, 27, 29, 48

science, 1, 14, 35, 45, 47, 48, 50 scientific theory, 38 side effects, 33 social behavior, 5 social change, 19 society, 50 spelling, 24, 27 stimulus, 3 structure, 24, 27, 28, 29 style, 5, 12

T

taxonomy, 3 teachers, 1, 19, 29, 49 teaching strategies, 19 techniques, 18 training, ix, xv, xvi, xvii, 2, 18, 20, 30,

31, 37, 39, 40, 43, 44, 48, 49, 50 transmission, 29 transportation, 9 triggers, 30 tutoring, 29

V

validation, 7 variables, xvi, xvii, 2, 5, 7, 28, 37, 38 variations, 48 vein, 18, 19, 39, 50

W

waste, 48

Does Asking Pertinent Non-Academic Questions Make You a Better Researcher

Documents

Transcript of Does Asking Pertinent Non-Academic Questions Make You a Better Researcher