Gender & Concentration in the AMCS Major

Instrument Construction and Reliability Analysis

Group MembersMatt Kriesel - Research IntroductionCharlie Sandor - The Research PlanLucas Ganser - Selection of TopicDavid Ricksford - HypothesisAngie Willis - Survey QuestionsManley Peterson - Validity/Reliability

Purposes of ResearchPure (Basic Science) – To understand the nature

of mathematical thinking, teaching and learningApplied (Engineering) – To use the basic science

understandings to improve mathematics instruction

Questions that Research in Mathematics Education

Address:Theoretical perspectives for understanding

thinking, learning, and teachingDescriptions of aspects of cognitionExistence proofsDescriptions of consequences of various forms

of instruction

Theories and ModelsIn mathematics, theories are explicit, results are

obtained analytically, and models are precise approximations.

Theories and models in the sciences are always subject to revision and refinement.

Criteria for Evaluating Models and Theories

Descriptive powerExplanatory powerPredictive powerScope

Rigor and specificityFalsifiabilityReplicabilityMultiple sources of

evidence

Evaluating Models and Theories

Descriptive power – capacity of a theory to capture “what counts.”

Explanatory power – providing explanations of how and why things work.

Predictive power – test of theory that can specify results in advance of their taking place.

Scope – range of phenomena covered by the theory

Evaluating Models and Theories (continued)

Rigor and Specificity – specifying a set of objects and relationships among the model or theory

Falsifiability – for making predictions whose accuracy can be tested empirically.

Replicability – ability of independent researchers to go through the same body of data and compare results.

Multiple sources of evidence – the more independent sources of confirmation, the more robust a finding is likely to be.

Last Semester BackgroundLast semester was spent acquiring the

background information we needed to perform an experiment.

This semester we implemented our knowledge gained from last semester into a survey instrument.

The Research Plan

What is a Research Plan?A detailed description of a proposed study

Justification for the studyDetailed description of the steps that will be followedInformation about the analysis of the collected data

A good research plan will help the researcher save time provide structure for the studyreduce the probability of costly mistakesresult in higher quality research

General Considerations in a Research Plan

The ethics of conducting researchThe legal restrictions on access to recordsStrategies for achieving and maintaining

cooperation

The Ethics of ResearchWhile doing research, there are many ethical

considerations to be followed, the two main rules being:

The participants should not be harmed in any wayThe participants privacy rights are maintained

The government has developed laws to protect research participants from harm and invasion of privacy

The National Research Act of 1974 requires that proposed research activities involving humans subjects be reviewed and approved by an authorized group in an institution.

The Family Educational Rights and Privacy Act of 1974 protects the privacy of the educational records of students.

Gaining Entry to the Research Site

In order to do research, the participation of many people is required.

The first step is getting permission from the necessary group or organizations where you are going to do the research.

In our study we had to receive permission from the Institutional Research Board(IRB) here on campus. The IRB is an organization that protects the rights of the participants in the study, and makes sure no ethical issues will be involved. The only problem that we had was that they wanted us to put the following statement on top of our survey “Informed Consent: Participation in this survey is strictly voluntary and confidential.”

Gaining Entry to the Research Site (continued)

Once formal permission is granted for the study, the researcher must get the participation of participants in the study.

Components of the Research Plan

Introduction SectionMethod SectionTime ScheduleBudget (if appropriate)

Introduction SectionThis section should be written using terms that

are of common usage, and includes:A statement of the topicA review of the related literatureA statement of the hypothesis

Method SectionA description of the participants including

The number, source, and characteristics of the sample.

Where the sample was drawn

What instrument will be used in the study and how it will be developed

Method Section (continued)

The procedure that will be followed in conducting the research from beginning to end, including:

A description of the technique to be used in selecting the sample

Exactly what is going to occur in the studyAny identified assumptions and limitationsIt should be as detailed as possible

Method Section (continued)

The last part of the methods section should also include information on data analysis, which includes a description of the technique that will be used to analyze the study data.

Time ScheduleThe major activities of the research should be listed in a

time schedule, with their anticipated completion dates.The researcher needs to

Allow for more time to complete the study than she thinksPlan for down timeSet the finishing date earlier than the final deadline for

completion

Summary of Research PlanThe Research Plan is important to any study:

it helps keep the research ethicalit helps to keep participation upand overall it helps the researchers keep on track

and on schedule.

Selection and Definition of a Topic

Identifying a Topic to Research

The first step in selecting a research topic is to identify a general area that is related to your area of expertise and is of particular interest to you.

There are three main sources of research topics: theory, personal experience, and replication

TheoryTheories are composed of organized bodies of

concepts, generalizations, and principles. Research studies commonly test or examine particular aspects of a theory to determine its applicability.

Personal ExperienceA researcher’s personal experiences and

concerns often lead to useful and personally rewarding studies.

Commonly asked questions such as, “Why does that happen?” and “How would a different group respond to this approach?” can provide rich sources of topics if followed up.

ReplicationReplication is repeating an existing study. It is generally expected that some features of the

replication (e.g., sample, instruments) will differ from the original study.

Identifying a TopicOnce an initial topic is identified, it usually needs to be

narrowed and focused into a manageable topic to study.Qualitative and quantitative research often differ in the

timing of narrowing their topics. Quantitative topics are narrowed as quickly as possible.Qualitative topics are usually delayed until time is spent

in the setting.

Characteristics of Good Topics

A good problem has theoretical or practical significance; its solution should contribute in some way to the improvement of the educational process.

When writing a topic statement, you should indicate the variables of interest, the specific relationship between those variables that is to be investigated, and, ideally, the type of participants involved.

Our Chosen TopicWhy do UW-Stout students choose the

concentrations they do in the Applied Mathematics and Computer Science major?

Formulation and Statement of a Hypothesis

HypothesisA hypothesis is a statement of the researcher’s

prediction or guess of the relation that exists among the variables being investigated.

The researcher does not set out to prove his or her hypothesis, but rather, collects data to determine whether the hypothesis is supported or not by the data.

Formulation of a Hypothesis

• General ideas about a hypothesis:• It should be a statement relating variable

phenomena to a predicted result.• It should be a statement which can be proven or

disproved. • Multiple hypotheses strengthen the technique. • Biased hypotheses must be adequately identified

and carefully watched.

Characteristics of a Hypothesis

It is based on sound reasoning.It provides a reasonable explanation for the

predicted outcome.It clearly states the expected relationship

between defined variables.It is testable within a reasonable time frame.

Types of HypothesesInductive – a generalization made from a number

of observations.Deductive – derived from theory and is aimed at

providing evidence that supports, expands, or contradicts aspects of a given theory.

Research – states the expected relationship (or difference) between two variables.

Types of Hypotheses (continued)

Non-directional – indicates that a relationship or difference exists but does not indicate the direction of the difference.

Directional – indicates that a relationship or difference exists and indicates the direction of the difference.

Null – there is no significant relationship (or difference) between variables.

Hypothesis Testing

The two basic principles of hypothesis testing: The formulation of two mutually exclusive

hypothesis statements that, together, exhaust all possible outcomes.

The testing of these so that one is necessarily accepted and the other rejected.

Our Hypotheses1. We believe a stronger presence of women on the AMCS

faculty would encourage female students to enroll.2. We believe women might be discouraged from trying

Software Development, because it is competitive.3. We believe teachers and advisors do not motivate

female students towards Software Development as much as they motivate male students.

Our Hypotheses (continued)

4. We believe remunerative (financial) aspects might be more important to men than to women.

5. We believe students who select the Business Management concentration are more likely to rank the social dimensions of an occupation more highly than those students in Software Development.

Testing Our HypothesesWhether you are testing hypotheses or seeking

answers to questions, you must decide on an instrument to collect your data.

Our instrument was a survey made up of ten questions, pertaining to each of our hypotheses.

What Our Survey Questions Were

Questions Generated From our Hypotheses

We chose to use a Likert scale for our survey. A Likert scale asks participants to respond to a series of statements by indicating whether they strongly disagree, disagree, are neutral, agree, or strongly agree with each statement. Each answer in a Likert scale can be assigned a point value which helps in the data analysis process.

DemographicsFor demographic information we asked each survey participant

their gender. We also asked for their choice of concentration from these choices:

• Software Development• Business Management• Actuarial Science

The survey participants were instructed that if they had not yet chosen a concentration to circle the one they were leaning toward. Also if a participant’s concentration was not listed we asked them to select the one closest to their concentration.

While writing these statements we:

• Avoided words that forced the survey participants to interpret them

• Avoided statements that might be offensive to some of the participants

• Worked to make the statements clear and accurate towards what we were trying to test

The following slides will show the survey statements we generated according to our hypotheses.

Hypothesis:

We believe a stronger presence of women on the AMCS faculty would encourage female students to enroll.

Survey Statements:

• The fact that the Software Development concentration is mostly male dominated strongly influenced my choice of concentration.

• MSCS teacher gender heavily influenced my choice of concentration.

• With a woman program director, Software Development is a more appealing concentration to me.

Hypothesis:

We believe remunerative (financial) aspects might be more important to men than to women.

Survey Statements:

• A high paying job is very important to me.

• I chose my concentration because of its potential job opportunities.

Hypothesis:

We believe women might be discouraged from trying Software Development, because it is competitive.

Survey Statements:

• I enjoy working in a competitive environment.

• If given a choice, I’d prefer taking a more challenging class, as opposed to a less challenging one.

Hypothesis:

We believe students who select the Business Management concentration are more likely to rank the social dimensions of an occupation more highly than those students in Software Development.

Survey Statement:

• A lot of social interaction is important in my career path.

Hypothesis:

We believe teachers and advisors do not motivate female students towards Software Development as much as they motivate male students.

Survey Statements:

• The concentration I chose is the one I was most heavily pushed toward by advisors, faculty and program directors.

• My choice of concentration was heavily influenced by positive experiences in introductory courses.

Validity and Reliability

Background on ValidityValidity is one of the most important characteristics a

measuring instrument can possess.Since all researchers test their subjects for a reason,

they want to make sure they get valid results.The validity of the instrument allows the researchers to

draw definite conclusions from the results.

Background on Reliability Reliability is essentially the dependability of an

instrument to test what it was designed to test. The more reliable an instrument is, the more

confident researchers can be that the scores obtained from the test would be the same if the test were given again to the same takers.

Estimating Reliability for Our Survey

After inputting all the survey data information into SPSS (which is a statistical computer program for Windows), we calculated the Cronbach’s alpha.

Cronbach’s alpha is an internal consistency estimate. The formula is:

alpha = N*P/(1+P(N-1))N = number of items on testP = average of the correlation between each pair of items on the test

The Main Advantage of Cronbach’s alpha

It has been proven that Cronbach’s alpha is a lower bound estimate of the reliability for any scale of N items, thus making it an ideal conservative approach to estimating reliability.

How to use Cronbach’s alpha in SPSS for WindowsHave your data on the screenClick on Analyze in the toolbarSelect Scale -> Reliability AnalysisPick which items you want to do the analysis onPick the alpha modelClick OK

A Screenshot of SPSS in Action

Our Cronbach’s alpha from SPSS

Reliability Analysis using Cronbach’s alphaReliability Coefficients

Number of Cases = 101Number of Items = 10

Alpha = .6318 or about 63% reliable

Our Thoughts on our Survey’s Cronbach’s alphaWe would have liked a higher alpha than we received,

but we believe a 63% reliability is respectable for our first instrument.

Some ways we could improve our reliability are to ask better questions and possibly have more of the same type of question.

More time would have been welcomed, since we spent a good portion of this semester laying down some necessary groundwork.

Continuation of AnalysisComing up next, our sister group will elaborate

more on the analysis of the data we received from our survey.

ReferencesAirasian, Peter and Gay, L.R. "Educational Research, Competencies for Analysis and Application", 6th edition

McKnight, Gagid, and Murphy. "Mathematics Education Research: A guide for the Research Mathematician.”