Post on 25-Feb-2016
description
Gender & Concentration in the AMCS Major
Instrument Construction and Reliability Analysis
Group MembersMatt Kriesel - Research IntroductionCharlie Sandor - The Research PlanLucas Ganser - Selection of TopicDavid Ricksford - HypothesisAngie Willis - Survey QuestionsManley Peterson - Validity/Reliability
Purposes of ResearchPure (Basic Science) – To understand the nature
of mathematical thinking, teaching and learningApplied (Engineering) – To use the basic science
understandings to improve mathematics instruction
Questions that Research in Mathematics Education
Address:Theoretical perspectives for understanding
thinking, learning, and teachingDescriptions of aspects of cognitionExistence proofsDescriptions of consequences of various forms
of instruction
Theories and ModelsIn mathematics, theories are explicit, results are
obtained analytically, and models are precise approximations.
Theories and models in the sciences are always subject to revision and refinement.
Criteria for Evaluating Models and Theories
Descriptive powerExplanatory powerPredictive powerScope
Rigor and specificityFalsifiabilityReplicabilityMultiple sources of
evidence
Evaluating Models and Theories
Descriptive power – capacity of a theory to capture “what counts.”
Explanatory power – providing explanations of how and why things work.
Predictive power – test of theory that can specify results in advance of their taking place.
Scope – range of phenomena covered by the theory
Evaluating Models and Theories (continued)
Rigor and Specificity – specifying a set of objects and relationships among the model or theory
Falsifiability – for making predictions whose accuracy can be tested empirically.
Replicability – ability of independent researchers to go through the same body of data and compare results.
Multiple sources of evidence – the more independent sources of confirmation, the more robust a finding is likely to be.
Last Semester BackgroundLast semester was spent acquiring the
background information we needed to perform an experiment.
This semester we implemented our knowledge gained from last semester into a survey instrument.
The Research Plan
What is a Research Plan?A detailed description of a proposed study
Justification for the studyDetailed description of the steps that will be followedInformation about the analysis of the collected data
A good research plan will help the researcher save time provide structure for the studyreduce the probability of costly mistakesresult in higher quality research
General Considerations in a Research Plan
The ethics of conducting researchThe legal restrictions on access to recordsStrategies for achieving and maintaining
cooperation
The Ethics of ResearchWhile doing research, there are many ethical
considerations to be followed, the two main rules being:
The participants should not be harmed in any wayThe participants privacy rights are maintained
The government has developed laws to protect research participants from harm and invasion of privacy
The National Research Act of 1974 requires that proposed research activities involving humans subjects be reviewed and approved by an authorized group in an institution.
The Family Educational Rights and Privacy Act of 1974 protects the privacy of the educational records of students.
Gaining Entry to the Research Site
In order to do research, the participation of many people is required.
The first step is getting permission from the necessary group or organizations where you are going to do the research.
In our study we had to receive permission from the Institutional Research Board(IRB) here on campus. The IRB is an organization that protects the rights of the participants in the study, and makes sure no ethical issues will be involved. The only problem that we had was that they wanted us to put the following statement on top of our survey “Informed Consent: Participation in this survey is strictly voluntary and confidential.”
Gaining Entry to the Research Site (continued)
Once formal permission is granted for the study, the researcher must get the participation of participants in the study.
Components of the Research Plan
Introduction SectionMethod SectionTime ScheduleBudget (if appropriate)
Introduction SectionThis section should be written using terms that
are of common usage, and includes:A statement of the topicA review of the related literatureA statement of the hypothesis
Method SectionA description of the participants including
The number, source, and characteristics of the sample.
Where the sample was drawn
What instrument will be used in the study and how it will be developed
Method Section (continued)
The procedure that will be followed in conducting the research from beginning to end, including:
A description of the technique to be used in selecting the sample
Exactly what is going to occur in the studyAny identified assumptions and limitationsIt should be as detailed as possible
Method Section (continued)
The last part of the methods section should also include information on data analysis, which includes a description of the technique that will be used to analyze the study data.
Time ScheduleThe major activities of the research should be listed in a
time schedule, with their anticipated completion dates.The researcher needs to
Allow for more time to complete the study than she thinksPlan for down timeSet the finishing date earlier than the final deadline for
completion
Summary of Research PlanThe Research Plan is important to any study:
it helps keep the research ethicalit helps to keep participation upand overall it helps the researchers keep on track
and on schedule.
Selection and Definition of a Topic
Identifying a Topic to Research
The first step in selecting a research topic is to identify a general area that is related to your area of expertise and is of particular interest to you.
There are three main sources of research topics: theory, personal experience, and replication
TheoryTheories are composed of organized bodies of
concepts, generalizations, and principles. Research studies commonly test or examine particular aspects of a theory to determine its applicability.
Personal ExperienceA researcher’s personal experiences and
concerns often lead to useful and personally rewarding studies.
Commonly asked questions such as, “Why does that happen?” and “How would a different group respond to this approach?” can provide rich sources of topics if followed up.
ReplicationReplication is repeating an existing study. It is generally expected that some features of the
replication (e.g., sample, instruments) will differ from the original study.
Identifying a TopicOnce an initial topic is identified, it usually needs to be
narrowed and focused into a manageable topic to study.Qualitative and quantitative research often differ in the
timing of narrowing their topics. Quantitative topics are narrowed as quickly as possible.Qualitative topics are usually delayed until time is spent
in the setting.
Characteristics of Good Topics
A good problem has theoretical or practical significance; its solution should contribute in some way to the improvement of the educational process.
When writing a topic statement, you should indicate the variables of interest, the specific relationship between those variables that is to be investigated, and, ideally, the type of participants involved.
Our Chosen TopicWhy do UW-Stout students choose the
concentrations they do in the Applied Mathematics and Computer Science major?
Formulation and Statement of a Hypothesis
HypothesisA hypothesis is a statement of the researcher’s
prediction or guess of the relation that exists among the variables being investigated.
The researcher does not set out to prove his or her hypothesis, but rather, collects data to determine whether the hypothesis is supported or not by the data.
Formulation of a Hypothesis
• General ideas about a hypothesis:• It should be a statement relating variable
phenomena to a predicted result.• It should be a statement which can be proven or
disproved. • Multiple hypotheses strengthen the technique. • Biased hypotheses must be adequately identified
and carefully watched.
Characteristics of a Hypothesis
It is based on sound reasoning.It provides a reasonable explanation for the
predicted outcome.It clearly states the expected relationship
between defined variables.It is testable within a reasonable time frame.
Types of HypothesesInductive – a generalization made from a number
of observations.Deductive – derived from theory and is aimed at
providing evidence that supports, expands, or contradicts aspects of a given theory.
Research – states the expected relationship (or difference) between two variables.
Types of Hypotheses (continued)
Non-directional – indicates that a relationship or difference exists but does not indicate the direction of the difference.
Directional – indicates that a relationship or difference exists and indicates the direction of the difference.
Null – there is no significant relationship (or difference) between variables.
Hypothesis Testing
The two basic principles of hypothesis testing: The formulation of two mutually exclusive
hypothesis statements that, together, exhaust all possible outcomes.
The testing of these so that one is necessarily accepted and the other rejected.
Our Hypotheses1. We believe a stronger presence of women on the AMCS
faculty would encourage female students to enroll.2. We believe women might be discouraged from trying
Software Development, because it is competitive.3. We believe teachers and advisors do not motivate
female students towards Software Development as much as they motivate male students.
Our Hypotheses (continued)
4. We believe remunerative (financial) aspects might be more important to men than to women.
5. We believe students who select the Business Management concentration are more likely to rank the social dimensions of an occupation more highly than those students in Software Development.
Testing Our HypothesesWhether you are testing hypotheses or seeking
answers to questions, you must decide on an instrument to collect your data.
Our instrument was a survey made up of ten questions, pertaining to each of our hypotheses.
What Our Survey Questions Were
Questions Generated From our Hypotheses
We chose to use a Likert scale for our survey. A Likert scale asks participants to respond to a series of statements by indicating whether they strongly disagree, disagree, are neutral, agree, or strongly agree with each statement. Each answer in a Likert scale can be assigned a point value which helps in the data analysis process.
DemographicsFor demographic information we asked each survey participant
their gender. We also asked for their choice of concentration from these choices:
• Software Development• Business Management• Actuarial Science
The survey participants were instructed that if they had not yet chosen a concentration to circle the one they were leaning toward. Also if a participant’s concentration was not listed we asked them to select the one closest to their concentration.
While writing these statements we:
• Avoided words that forced the survey participants to interpret them
• Avoided statements that might be offensive to some of the participants
• Worked to make the statements clear and accurate towards what we were trying to test
The following slides will show the survey statements we generated according to our hypotheses.
Hypothesis:
We believe a stronger presence of women on the AMCS faculty would encourage female students to enroll.
Survey Statements:
• The fact that the Software Development concentration is mostly male dominated strongly influenced my choice of concentration.
• MSCS teacher gender heavily influenced my choice of concentration.
• With a woman program director, Software Development is a more appealing concentration to me.
Hypothesis:
We believe remunerative (financial) aspects might be more important to men than to women.
Survey Statements:
• A high paying job is very important to me.
• I chose my concentration because of its potential job opportunities.
Hypothesis:
We believe women might be discouraged from trying Software Development, because it is competitive.
Survey Statements:
• I enjoy working in a competitive environment.
• If given a choice, I’d prefer taking a more challenging class, as opposed to a less challenging one.
Hypothesis:
We believe students who select the Business Management concentration are more likely to rank the social dimensions of an occupation more highly than those students in Software Development.
Survey Statement:
• A lot of social interaction is important in my career path.
Hypothesis:
We believe teachers and advisors do not motivate female students towards Software Development as much as they motivate male students.
Survey Statements:
• The concentration I chose is the one I was most heavily pushed toward by advisors, faculty and program directors.
• My choice of concentration was heavily influenced by positive experiences in introductory courses.
Validity and Reliability
Background on ValidityValidity is one of the most important characteristics a
measuring instrument can possess.Since all researchers test their subjects for a reason,
they want to make sure they get valid results.The validity of the instrument allows the researchers to
draw definite conclusions from the results.
Background on Reliability Reliability is essentially the dependability of an
instrument to test what it was designed to test. The more reliable an instrument is, the more
confident researchers can be that the scores obtained from the test would be the same if the test were given again to the same takers.
Estimating Reliability for Our Survey
After inputting all the survey data information into SPSS (which is a statistical computer program for Windows), we calculated the Cronbach’s alpha.
Cronbach’s alpha is an internal consistency estimate. The formula is:
alpha = N*P/(1+P(N-1))N = number of items on testP = average of the correlation between each pair of items on the test
The Main Advantage of Cronbach’s alpha
It has been proven that Cronbach’s alpha is a lower bound estimate of the reliability for any scale of N items, thus making it an ideal conservative approach to estimating reliability.
How to use Cronbach’s alpha in SPSS for WindowsHave your data on the screenClick on Analyze in the toolbarSelect Scale -> Reliability AnalysisPick which items you want to do the analysis onPick the alpha modelClick OK
A Screenshot of SPSS in Action
Our Cronbach’s alpha from SPSS
Reliability Analysis using Cronbach’s alphaReliability Coefficients
Number of Cases = 101Number of Items = 10
Alpha = .6318 or about 63% reliable
Our Thoughts on our Survey’s Cronbach’s alphaWe would have liked a higher alpha than we received,
but we believe a 63% reliability is respectable for our first instrument.
Some ways we could improve our reliability are to ask better questions and possibly have more of the same type of question.
More time would have been welcomed, since we spent a good portion of this semester laying down some necessary groundwork.
Continuation of AnalysisComing up next, our sister group will elaborate
more on the analysis of the data we received from our survey.
ReferencesAirasian, Peter and Gay, L.R. "Educational Research, Competencies for Analysis and Application", 6th edition
McKnight, Gagid, and Murphy. "Mathematics Education Research: A guide for the Research Mathematician.”