Evaluating the impact of Mars and Venus Effect on the use of an Adaptive Learning Technology for...

1July, 2016

Evaluating the impact of Mars and Venus Effect on the use of an Adaptive Learning Technology for Portuguese and Mathematics

Center of Excellence for Social TechnologiesComputing Institute (IC) - Federal University of Alagoas (UFAL)

[email protected] Joaquim, Ig Ibert Bittencourt, et al

The 16th IEEE International Conference on Advanced Learning Technologies - ICALT2016 – Austin, Texas

Brasil

Maceió

Maceió

Maceió, Alagoas, Brazil

6July, 2016





14th Century

http://media.linkjab.com/i/images/university-of-bologna-the-worlds-oldest-university.jpg

http://media.linkjab.com/i/images/university-of-bologna-the-worlds-oldest-university.jpg

14th Century

Few evolution in this area

1870 Paoli Classroom

http://sjvnoticias.com/sala-de-aula-ontem-e-hoje/ 2015

http://www.grmwebsite.com/blog/bid/67340/Welcome-to-the-21st-Century-Why-YOU-Should-Be-Marketing-on-the-Web 21st Century

| www.nees.com.br/enIg Ibert Bittencourt | [email protected]

https://onlineacademiccommunity.uvic.ca/jjohal93/category/blogs/

http://www.theaerodrome.com/forum/showthread.php?t=61895

http://www.theaerodrome.com/forum/showthread.php?t=61895http://andthatswhyyouresingle.com/2013/04/14/why-testing-a-man-will-almost-always-backfire/frustrated-woman-at-computer-3/

Gamification


Adaptive Techniques for Individualized Learning


Skills

Topics

Disciplines

[Vail et al., 2015]

Of all the learner characteristics that are known to influence the effectiveness of adaptive support, gender is one of the most widely recognized.


In several domains: Mathematics; Introduction to Computer Programming.

[Arroyo et al., 2011, 2013]; [Vail et al., 2015]

Gender difference in the use of intelligent educational technologies:

http://passofundo.notredame.org.br/estudantes-conquistam-a-2a-colocacao-em-olimpiada-de-programacao-de-computadores/

http://pt.123rf.com/photo_31784263_menina-da-escola-doce-latin-pouco-com-rabo-de-cavalo-cansado-e-entediado-com-o-computador-de-casa-ma.html


According to these studiesThere is a difference in:

Behavior; Attitude; Learning and; Emotion based on the gender.


http://blog.tricae.com.br/educacao/psicomotricidade-na-educacao-infantil/ http://colegiomarista.org.br/santamaria/diferenciais/para-mais-de-um-nivel


Cognitive levels: Conceptual and problem-based approaches; Affective assistance (motivation and engagement)



Gender Difference Studies: Usually performed as lab studies; Few studies investigated the gamification support; Only some studies about language course (but no Portuguese); No experimental study with ecological settings in Brazil; No model to estimate Math and Portuguese performance based on

gender.

http://blog.tricae.com.br/educacao/psicomotricidade-na-educacao-infantil/ http://colegiomarista.org.br/santamaria/diferenciais/para-mais-de-um-nivel



Our Study

Mars and Venus Effect

Different variables: Learning performance (Portuguese and

Mathematics); Age; Location area; Adaptive learning technology.


Investigate learning performance (Math and Portuguese); The gamification support; Experimental study with ecological settings; Propose a model to estimate Math and Portuguese

performance.


Instrument: A Gamified Intelligent Tutoring System

100 5000

+30.000

NUMBER OF USERS

STUDENTS

48%FASTER More learning

27%

Game elements of MeuTutor


Student solving a recommended problem for him in MeuTutor


5th Grade

School D

School C

School B

School A

5th Grade

5th Grade

5th Grade

9th Grade

9th Grade

9th Grade

9th Grade


School C

School D

School A

School B

Experimental group Control group


School A

5th Grade 9th Grade

School B

5th Grade 9th Grade

Experimental group (with the use of ITS gamified)


School C

5th Grade 9th Grade

School D

5th Grade 9th Grade

Control group (without the use of ITS gamified)


Timeline


Intervention models

Teacher

Intervention models

Integrated class instruction Homework

After-class instruction



Results

Participants

191students

87 (45.55%) Male

104 (54.45%)Female

Between 9 and 21 years old (mean = 13.8, sd = 2.01)


the use of an adaptive learning technology improves student’s performance in Mathematics and Portuguese for both male and female students (p-Value < .05).


The improvement of male student’s performance is more significant (p-Value < .05)

Math Portuguese

39.1%43.4%

19.7%24.6%

Male Female


When we focus on technology, male students had better performance in mathematics (21%), but no significant difference in Portuguese.


Beta regression model

The response variables of the models are:

Ymath (Math performance);

Yport (Port performance).

The mean Yport or Ymath,



Where:

is the linear predictor;

is the sample size;

is the individual;

is an unknown vector parameter;

are fixed covariates .

Can be written as:


Almost all covariates are significant at the 5% nominal level (p-Value < .05), except Gender for

the performance model in Portuguese.


Parameter Estimates and P-value for *General* Models. Full dataset.



The systematic components of the models were defined by the following mathematical expressions:



Other six models, only considering the significant covariates


In this case:

If a model is correctly specified we expect that the residuals are randomly scattered around zero in the plots of residuals against observations index.

The residuals graphical analysis

It is essential to validate the aforementioned conclusions


The normal probability plot with generated envelope:

Can be used even when the empirical distributions of the residuals are not normal;

If the model fits well, we expect that most of the residuals fall within the band;

Are based on the differences between the observed responses (y) and its fitted value ().

Another residuals graphical


In figures

See: a, e, I, m - c, g, l, o

It is possible to notice that the residuals are randomly scattered around zero (no systematic features) in all plots of the combined residual against the observations index, for both Math and Portuguese models.


However,...

For the normal probability plots there is a slight evidence of misspecification of two models for the Mathematics performance, the “Only Using MeuTutor” and the “Female” models, suggesting the need for other explicative variables, possibly elements of the MeuTutor; see Figures (f, n).



Six models, only considering the significant covariates


Such as:

Receptiveness to technology

Preference of pedagogical approach

The meaning of optimal experience

Gamified Approach

Stereotype Threat

Possible Explanations to Technology and Female Models:


62


Psychological Mediator

Affective Mechanisms

Cognitive Mechanisms

Motivational Mechanisms

Stereotype Threat

Less Performance

Anxiety

Individual Tendencies

Apprehension

Self-efficacy

Dejection

Self-handicapping

Effort/Motivation

Vigilance

Cognitive Load

Thought suppression Negative ThinkingMind-wandering

63


64


Different Gamification

Elements

Less Improvement

Implicit Stereotype

Threat

New Hyphotesis

Confirmed the hypothesis of gender’s influence on adaptive learning environments, by presenting its reliability with other research settings and instruments.

Proposition of models to estimate performance:

To sum up

New hypothesis raised to explain the Mars and Venus Effects related to the design of gamified educational systems.


ReferencesIvon Arroyo, Beverly P. Woolf, David G. Cooper, Winslow Burleson, Kasia Muldner. "The Impact of Animated Pedagogical Agents on Girls’ and Boys’ Emotions, Attitudes, Behaviors and Learning". Advanced Learning Technologies (ICALT), 2011 11th IEEE International Conference on. Doi: 10.1109/ICALT.2011.157.

Ivon Arroyo, Winslow Burleson, Minghui Tai, Kasia Muldner, Beverly Park Woolf. “Gender differences in the use and benefit of advanced learning technologies for mathematics”. Journal of Educational Psychology, Vol 105(4), Nov 2013, 957-969. http://dx.doi.org/10.1037/a0032748.

Alexandria Katarina Vail, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester."The Mars and Venus Effect: The Influence of User Gender on the Effectiveness of Adaptive Task Support". 23rd International Conference, UMAP 2015, Dublin, Ireland, June 29 -- July 3, 2015. Proceedings. Doi: 10.1007/978-3-319-20267-9_22.

Ferrari, S.L.P.; Cribari-Neto, F. (2004) “Beta regression for modelling rates and proportions”, Journal of Applied Statistics.


ReferencesEspinheira, P.L., Silva, C.M. Silva, A.O. (2015). “Prediction Measures in Beta Regression Models”. arXiv:1501.04830.

Gneezy, Uri, and Aldo Rustichini. "Gender and competition at a young age." The American Economic Review 94.2 (2004): 377-381.

Niederle, Muriel, and Lise Vesterlund. "Gender and competition." Annu. Rev. Econ. 3.1 (2011): 601-630.

Kim, Jung Tae, and Won-Hyung Lee. "Dynamical Model and Simulations for Gamification of Learning." International Journal of Multimedia and Ubiquitous Engineering 8.4 (2013): 179-190.

Koivisto, Jonna, and Juho Hamari. "Demographic differences in perceived benefits from gamification." Computers in Human Behavior 35 (2014): 179-188.

Paiva, Ranilson O. Araújo, et al. "Improving pedagogical recommendations by classifying students according to their interactional behavior in a gamified learning environment." Proceedings of the 30th Annual ACM Symposium on Applied Computing. ACM, 2015.


Maceió

Maceió, Alagoas, Brazil

Thank you!Muito Obrigado!

Questions???

72July, 2016





Evaluating the impact of Mars and Venus Effect on the use of an Adaptive Learning Technology for...

Education

Transcript of Evaluating the impact of Mars and Venus Effect on the use of an Adaptive Learning Technology for...