Welcome to the ALSET webinar!nus.edu.sg/alset/wp-content/uploads/2020/07/Use... · Welcome to the...

Welcome to the ALSET webinar!

You will be muted on entry, but you may keep your

camera on.

This webinar will be recorded. You can edit your name to stay

anonymous.

For questions use the Chat panelQ&A at the end of the session

(email is possible)

References : http://nus.edu.sg/alset/2020/07/10/adl-webinar/

http://nus.edu.sg/alset/2020/07/10/adl-webinar/

An Exploratory Tour of ALSET Data Lake: Towards Better Teaching and Research

TAN TECK KIANG, ALSETALEX BIOTTEAU, ALSET

Contents

•What is ALSET?•An Overview of ALSET Data Lake (ADL)•ADL Data Analytics Example – Field of Study

Mismatch, GES 2016•Demonstration - ADL•Possible Data Analytics & Research Projects for

Better Teaching and Research using ADL•References

What it ALSET?

NUS Institute for Applied Learning sciences and Educational

Technologies

● Interdisciplinary Institute

● Created in 2016

● Educational Research

Management & staff

InterdisciplinarityAffiliates’ Main Disciplines

8

Data LakeDATA

FUNDINGSeed grants,

information on extramural grants

SUPPORTLACE & IRB

applications,statistician

Collaboration,institutions

COMMUNICATIONSeminar,

newsletter,Internet

CONTACTS

Resources for Faculty

ALSET Data Lake




Main tables of educational data70 Tables in ADL (May 2020)

SurveyData

Survey QuestionGraduate Employment Survey 2015+

Main surveys

SurveyData

Survey QuestionOrientation Camp Survey 2017

Student Feedback Survey Feedback for Module

Feedback for Instructors

SurveyData

Orientation QuestionFreshman Entry Survey 2018

Survey data

Survey Questionnaire

Graduate Employment Survey

Examples: GES & OCSOrientation Camp Survey

Section Description

A Background

B Employment Activity Status

C Employment Particulars

D Job Application

E Future Studies

F NUS Experience

G Demographic

Section Description

A Faculty

B Satisfaction

C Motivation

D About OC

E Social Networking

F Opinion About OC

G About OC (con’d)

H Respondent’s Background

I About Survey and Contact

2015 and 2016

Survey data


for 2017

Tokenised Student TablesPersonal

Data

Data ItemsStudent TokenGender

Year of BirthMarital Status

CountryResidency

Job Title

…

Program Enrolment

Data ItemsStudent TokenProgram

FacultyDepartmentTerm (e.g.1910)Start Data

End Date

Partner UniversityProgram Category

…

Module Enrolment

Data ItemsStudent TokenModule

CourseFinal GradeFinal Mark

Grading BasisGrade Points per UnitModule CreditEarned Credit

Class Group

...

Term and Grade History

Data ItemsStudent TokenTerm

Term GPACumulative GPSIinstitution

Academic Career

Career Number

External Test Score

Data ItemsStudent TokenTest Date

Test DescriptionTest ComponentTest Score

Test ID

Term and Grade History

Data ItemsStudent Token

TermTerm GPACumulative GPS

Iinstitution

Academic Career

Career Number

…

From 2005

Co-Curricular Activity Table

Profile Events Positions Involvement

Student Token

Country

City

State

Classification

Community

Portal

…

Student Token

Event ID

Event Description

Event CategoryEvent DescriptionEvent Title

Event Location

…

Student Token

Position Name

Position Start Date

Position End Date

...

Student Token

Community ID

Community Name

Start Data

End Date

…

LMS Audit Log

Data ItemsStudent Token

Session ID

IP Address

Day (e.g. 2019-10-16)

Date Time

Action

Etc.

View Download Files page

View Media from Multimedia

View class groups

Search Module

Saving Assessment Answer

Module Overview

View Unread Announcements

View Modules

File download

Start Assessment button

Submit Assessment button

Actiontoken1

token1

token1

token1

token1

token1

token1

token1

token1

token1

Coming & Developing• LumiNUS audit logs

• IMMS (Continuous Assessment grades)

• Graduate admissions (GDA2)

• Alumni

• Library search logs

• Class attendance system

• Internship offerings and placements

• …

ALSET Data Lake




Survey


Graduate Employment Survey 2016

Data Structure

Question No, Short and Long Description

First Ten Observations First Choice of Study and Field of Study Mismatch

FOSMatch

FOSMismatch

Field of Study Occupation

Law Lawyer / Solicitor

Accountancy Accountant

What is Field of Study (FOS) Mismatch?

Law Financial Adviser

Accountancy Social Worker

Is the nature of your present job related to your course of study of NUS?

(1) yes, I am employed in a job related to my course of study(2) Yes, I am employed in a job partially related to my course of study(3) No, I am employed in a job not related to my course of study

(1) Match (2) Partial(3) Mismatch

GES 2016Field of Study After WorkingField of Study Mismatch

Income Earning and Field of Study

[Some information masked for confidentiality reason]

Mismatch Affect Income Earning?

FOS Match earned higher income.

Income earned differ by FOS

Gender Effect

Regression Model (OLS)


Mixed-Effect Model - Mismatch Affect Income Earning?

(1) 43.1% of Variation in Income is from Faculty

(2) Male Earned Higher After Control for FOS Mismatch

(3) Specifying Gender RandomGender Effect Disappears

(4) Specifying FOS RandomFOS Effect Disappears

DropIncrease


ALSET Data Lake




https://jhub.nus.edu.sg/hub/login

ALSET Data Lake




Possible Data Analytics: Teaching and Research using ADL• ADL as A Sampling Frame• Study Directly from ADL

1. Research Study Based on ADL (Second Year Economic)2. Understanding Student Progress (Longitudinal Data Analysis)3. Propensity Score Analysis (Matching Students for Course Comparison) 4. Classification of Student Learning Pattern (Sequence Analysis)

• Integrating Assessment / Survey into ADL 1. Sleep Habit Study2. Cognitive Diagnostic Model / Item Response Theory (Assessment and

Cognitive Diagnostic)

ADL as aSampling Frame

ADL

StudentsTook

Second YearEconomics

Subjects

Targeted Population

200Students

Sample

Survey /Assessment

Preliminary Analysisfrom ADL

Back to ADL for Further Analysis

Random Sampling

No. of Students

By Module By Faculty


Research Study and Analysis Based on ADL

Module Personal Program Enrol

Pre-Entry Qualification CCA

GES

Identify the Modules(1) Module(2) Year(3) Grade(4) Mark

(1) Gender(2) Residency(3) Age(4) Live in Hostel

Year Award (Pre-Entry)

Faculty No of CCA Activities

Monthly IncomeGrade by Module

over Year

More Male or Female?

CCA by Module

Longitudinal Data AnalysisUsing ADL

Student 1

Year 1 Year 3Year 2

Student 3,420

Year 1 Year 3Year 2…

…

Quantile Regression Using ADL

Initial Higher GradeLower Growth

Initial Lower GradeHigher Growth

Unconditional Growth Model

𝑌!" = 𝛽#" + 𝛽$"𝑡!" + 𝑟!"𝛽#" = 𝛾## + 𝑢#"𝛽$" = 𝛾$# + 𝑢$"𝑟!"~𝑁 0, 𝜎%

𝑢#"𝑢$" ~𝑁 0

0,𝜏##% 𝜏#$𝜏#$ 𝜏$#%

r 𝑢#" , 𝑢$" 𝜏#$ =-0.71

𝑌!" = 30.42 + 0.17 𝑡!" + 𝑟!"

The Intraclass Correlation = 0.3547Variation in Grade 35.47% is between student64.54% is over time.

Increase in grade over time but small

Initial grade and growth grade are negatively related

Understanding Student

Progression

ADL as a Source of Carrying Out Experimental Design

Fisher’s Randomized Experiment Attend Language Classes

Did not Attend Language Classes

It is designed to answer specific questions about a particular treatment strategy

Propensity Score

Gender Country CCA…

Attended Not Attended

Propensity Score Analysis

Performance Score minus Performance Score

Matching

Classification of Student Learning Pattern & Sequence

Wifi1

Wifi2

Wifi3

Wifi4

Wifi5

Wifi6

Wifi7

Wifi8

Wifi9

Wifi10

Wifi1

Wifi2

Wifi3

Wifi4

Wifi5

Wifi6

Wifi7

Wifi8

Wifi9

Wifi10

…

Learning Pattern

IVLE 1

IVLE 2

IVLE 3

IVLE 4

IVLE 5

IVLE 6

IVLE 7

IVLE 8

IVLE 9

IVLE 10

IVLE 1

IVLE 2

IVLE 3

IVLE 4

IVLE 5

IVLE 6

IVLE 7

IVLE 8

IVLE 9

IVLE 10

On-Line Access Pattern

Sequence AnalysisOptimal Matching

0𝑑(2,1) 0𝑑(3,1) 𝑑(3,2) 0⋮ ⋮ ⋮

𝑑(𝑛, 1) 𝑑(𝑛, 2) ⋯ ⋯ 0

A Matrix Shows How SimilarAre The Sequences of These Students.

Classification

e.g. Cluster

AnalysisExplanation (from ADL)ln

𝜋!"𝜋!#

= 𝛽$ + 𝛽%xi1 + 𝛽&xi2+. . . +𝛽'"xik

Multinomial RegressionFactors Influencing Learning

Sequence to Form Learning Grouping

Learning Pattern Affecting Academic Outcomes

Sleep Habit Survey 2019PI Joshua J. Gooley

Sleep Habit

SurveyADL

ADLIVLE

ADLWifi

From Survey to ADL

From AssessmentUsing Cognitive Diagnostic Model

to Understand Student SkillsLink to ADL for Further Analysis CDM /

IRT

ADL Further Analysis

Student Item 1 Item 2 … Item 19 Item 201 0 0 … 1 12 0 1 … 1 13 1 1 … 0 0

… … … … … …199 1 1 … 0 0200 1 1 … 0 0

Assessment200 Students + 20 Items

Skills Probability𝛼! 0.58𝛼" 0.79𝛼# 0.72𝛼$ 0.69𝛼% 0.60𝛼& 0.79𝛼' 0.81𝛼( 0.82

𝛼! 𝛼" 𝛼# 𝛼$ 𝛼% 𝛼& 𝛼' 𝛼(𝛼! 1 0.35 0.44 0.69 0.64 0.52 0.14 0.42𝛼" 0.35 1 0.42 0.32 0.19 0.50 0.92 0.69𝛼# 0.44 0.42 1 0.37 0.60 0.38 0.40 0.39𝛼$ 0.69 0.32 0.37 1 0.55 0.61 0.34 0.32𝛼% 0.64 0.19 0.60 0.55 1 0.51 0.18 0.33𝛼& 0.52 0.50 0.38 0.61 0.51 1 0.57 0.43𝛼' 0.14 0.92 0.40 0.34 0.18 0.57 1 0.63𝛼( 0.42 0.69 0.39 0.32 0.33 0.43 0.63 1

Number of Skills Mastered Skills Mastered Pattern

Guessing and Non-Slipping

Individual Diagnostic Report

Relationship of Skills

ALSET Data Lake




References

1. Package RODBC – 11 May 20202. Introduction to R (Version 2, 8 June 2020)3. ADL Data Management using R (1 June 2020)

Thank you and do keep in touch with us

Slides: http://nus.edu.sg/alset/2020/07/10/adl-webinar/

ALSET Website: http://www.nus.edu.sg/alset

Alex: [email protected]: [email protected]

Teck Kiang: [email protected]

http://nus.edu.sg/alset/2020/07/10/adl-webinar/

http://www.nus.edu.sg/alset

Welcome to the ALSET webinar!nus.edu.sg/alset/wp-content/uploads/2020/07/Use... · Welcome to the...

Documents

Transcript of Welcome to the ALSET webinar!nus.edu.sg/alset/wp-content/uploads/2020/07/Use... · Welcome to the...