PhD Mini Viva Talk
61
Mini-Viva Presentation Duygu Simsek [email protected] Supervisors: Prof. Simon Buckingham Shum, Dr. Anna De Liddo & Dr. Rebecca Ferguson Examiners: Prof. Steve Swithenby & Prof. Denise Whitelock The Open Science Laboratory 10/09/2013, Tuesday, The Open University Mini Viva Presentation
-
Upload
duygu-simsek -
Category
Education
-
view
1.221 -
download
2
description
This is the presentation of my mini viva talk given to examiners who assess my PhD's 1st year following the probationary report. It is a summary of my research aims, what I have been doing since the beginning of my 1st year and my plans for the following years of the PhD
Transcript of PhD Mini Viva Talk
Mini-Viva Presentation Duygu Simsek
Supervisors: Prof. Simon Buckingham Shum, Dr. Anna De Liddo & Dr. Rebecca Ferguson
Examiners: Prof. Steve Swithenby & Prof. Denise Whitelock
The Open Science Laboratory
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Research Aim
• To investigate
• Whether or not computational techniques can automatically identify attributes of good scholarly writing
• What is the potential of these techniques for student essay analysis?
• How we can best feedback the results of such analysis in a way that learners can value to improve the quality of their writing.
• How educators can use these results for automatic or semi-automatic assessment of their students writing.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Good Scholarly Writing? Quality of Writing? • Signalled by the use of
metadiscourse markers in the text.
• Metadiscourse:
• Linguistic cues in the text
• Expresses a viewpoint, the problem, claim, argument, the evidence and the implications
• Engages the readers, and signals the writer's stance.
Italicised words are example metadiscourse markers
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Xerox Incremental Parser (XIP)
• Automatic processing of scientific documents
• Recognition of the rhetorically significant sentences
• 8 categories of Rhetorical Moves
• Background Knowledge
• Summarising
• Tendency
• Novelty
• Significance
• Surprise
• Open Question
• Contrasting Ideas
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Xerox Incremental Parser (XIP)
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Original Contribution of PhD
• Carrying XIP into the education field
• For professional scientific articles written by experienced researchers
• But now for analysis of student essays
• Hypothesis: An outcome of the XIP processed scientific documents can demonstrate the quality of the author’s written discourse; and therefore can be used to scaffold and assess scholarly writing.
• First in depth opportunity to
• Assess a state of the art language technology
• Integrate its services into software tools for academic writing
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Research Questions
• Main Research Question
• How can we support students’ scholarly writing skills to improve the quality of their writing through automated metadiscourse analysis?
• Sub-Question 1 • How reliable and sufficient are the automated discourse analysis tools
for finding good attributes of scholarly writing within student essays?
• Sub-Question 2 • To what extent is there a relation between the existences of various
kinds of argumentative discourse moves in student essays with final grades?
• Sub-Question 3 • To what extent automated metadiscourse analysis of discipline-
independent student essays can be used to provide formative feedback?
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Academic Writing
Where this research sits?
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Computational Linguistics
Academic Writing
Where this research sits?
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Learning Analytics
Computational Linguistics
Academic Writing
Where this research sits?
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Learning Analytics
Computational Linguistics
Academic Writing
Where this research sits?
Scientific Writing
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Learning Analytics
Computational Linguistics
Academic Writing
Where this research sits?
Scientific Writing
Rhetorical
Parsers
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Learning Analytics
Computational Linguistics
Academic Writing
Where this research sits?
Scientific Writing
Rhetorical
Parsers
Discourse Centric
Learning Analytics
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Learning Analytics
Computational Linguistics
Academic Writing
Where this research sits?
Scientific Writing
Rhetorical
Parsers
Discourse Centric
Learning Analytics
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not • Computational Linguistics:
Possible automated analysis of scientific & technical writing but barely deployed in educational context!
• Need: XIP output is not
educator/learner friendly.
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not • Computational Linguistics:
Possible automated analysis of scientific & technical writing but barely deployed in educational context!
• Need: XIP output is not
educator/learner friendly.
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not • Computational Linguistics:
Possible automated analysis of scientific & technical writing but barely deployed in educational context!
• Need: XIP output is not
educator/learner friendly.
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not • Computational Linguistics:
Possible automated analysis of scientific & technical writing but barely deployed in educational context!
• Need: XIP output is not
educator/learner friendly.
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not • Computational Linguistics:
Possible automated analysis of scientific & technical writing but barely deployed in educational context!
• Need: XIP output is not
educator/learner friendly.
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not • Computational Linguistics:
Possible automated analysis of scientific & technical writing but barely deployed in educational context!
• Need: XIP output is not
educator/learner friendly.
• Learning Analytics: Promising potential of automated, timely & formative feedback.
• Unresolved Question: “What does
analytics look like for a higher order skill such as communicating ideas in scientific writing, with a primary focus on assessing students?”
Literature Review Journey
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
• Academic Writing: Writers signal argumentative moves by using well-established patterns
• Debate on whether these
patterns are discipline independent or not • Computational Linguistics:
Possible automated analysis of scientific & technical writing but barely deployed in educational context!
• Need: XIP output is not
educator/learner friendly.
• Run XIP on essays from different disciplines
• Validate XIP in educational context
• If we can show there is a value for learners & educators then it has a potential for formative assessment of writing.
Pilot Work: XIP Dashboard
• Aim: Visualise XIP output
• Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 1
0/0
9/2
01
3, T
ues
day
, Th
e O
pen
U
niv
ersi
ty
Min
i Viv
a P
rese
nta
tio
n
Pilot Work: XIP Dashboard
• Aim: Visualise XIP output
• Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 1
0/0
9/2
01
3, T
ues
day
, Th
e O
pen
U
niv
ersi
ty
Min
i Viv
a P
rese
nta
tio
n
Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
Pilot Work: XIP Dashboard
• Aim: Visualise XIP output
• Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 1
0/0
9/2
01
3, T
ues
day
, Th
e O
pen
U
niv
ersi
ty
Min
i Viv
a P
rese
nta
tio
n
Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
Pilot Work: XIP Dashboard
• Aim: Visualise XIP output
• Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 1
0/0
9/2
01
3, T
ues
day
, Th
e O
pen
U
niv
ersi
ty
Min
i Viv
a P
rese
nta
tio
n
XIP Analysis All of the papers (66 LAK and 239 EDM papers 305 in total) were analysed using XIP, extracting 7847 sentences and 40163 concepts.
Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
Pilot Work: XIP Dashboard
• Aim: Visualise XIP output
• Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections? 1
0/0
9/2
01
3, T
ues
day
, Th
e O
pen
U
niv
ersi
ty
Min
i Viv
a P
rese
nta
tio
n
XIP Analysis All of the papers (66 LAK and 239 EDM papers 305 in total) were analysed using XIP, extracting 7847 sentences and 40163 concepts.
Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
Pilot Work: XIP Dashboard
• Aim: Visualise XIP output
• Question in mind: Can visual XIP analysis of literature help users to assess current state of the art in terms of trends, patterns, gaps, and connections?
XIP Output: Not learner friendly
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
XIP Analysis All of the papers (66 LAK and 239 EDM papers 305 in total) were analysed using XIP, extracting 7847 sentences and 40163 concepts.
Data Set Proceedings of the Learning Analytics and Knowledge (LAK) conferences and a journal special issue, and the Educational Data Mining (EDM) conferences and journal.
Pilot Work: XIP Dashboard
• XIP Dashboard is a set of visual analytics modules built on XIP output.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Pilot Work: XIP Dashboard
• XIP Dashboard is a set of visual analytics modules built on XIP output.
Paper Prototype & User Evaluation 6 user sessions were conducted with 1st year PhD students from OU
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Pilot Work: XIP Dashboard
• XIP Dashboard is a set of visual analytics modules built on XIP output.
Paper Prototype & User Evaluation 6 user sessions were conducted with 1st year PhD students from OU
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Pilot Work: XIP Dashboard
• XIP Dashboard is a set of visual analytics modules built on XIP output.
Paper Prototype & User Evaluation 6 user sessions were conducted with 1st year PhD students from OU
XIP Dashboard
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Dissemination of Work
• Various poster presentations & talks.
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
What are the Next Plans?
• Design refinements to the XIP Dashboard
• User evaluations
• XIP as an API, Web Service
• Integrate to software tools, XIP Dashboard
• Test XIP’s power on student essays
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Data Collection & Analysis What? When? How? Why?
Student Essays of OU’s S288
S288-12B (2012 Course)
• Analysis of student essays through XIP
• Comparison of XIP findings with final grades
• To see whether XIP can identify important parts of student essays
• To see whether or not we can correlate XIP results with final grades
Student Essays of OU’s S288
S288-13B (2013 Course)
Same as two above • Use of Google Docs for
collaboratively written report where we back up the revision history & analyse through XIP
Same as two above • To see whether XIP can reveal
interesting predictive patterns about the quality of the end document and the final grade.
Student Essays of OU’s S288
S288-14B (2014 Course)
Same as above • Develop software with XIP
Visual Analytics Dashboard integrated
• Get users’ reactions
Same as above • Analyses student essays &
provide real-time analytics of students’ essays as a feedback to students.
Student Essays (soft domains)
N/A Same as above Same as above • Test the discipline independency
of XIP
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
Data Collection & Analysis What? When? How? Why?
Student Essays of OU’s S288
S288-12B (2012 Course)
• Analysis of student essays through XIP
• Comparison of XIP findings with final grades
• To see whether XIP can identify important parts of student essays
• To see whether or not we can correlate XIP results with final grades
Student Essays of OU’s S288
S288-13B (2013 Course)
Same as two above • Use of Google Docs for
collaboratively written report where we back up the revision history & analyse through XIP
Same as two above • To see whether XIP can reveal
interesting predictive patterns about the quality of the end document and the final grade.
Student Essays of OU’s S288
S288-14B (2014 Course)
Same as above • Develop software with XIP
Visual Analytics Dashboard integrated
• Get users’ reactions
Same as above • Analyses student essays &
provide real-time analytics of students’ essays as a feedback to students.
Student Essays (soft domains)
N/A Same as above Same as above • Test the discipline independency
of XIP
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within
student essays?
Data Collection & Analysis What? When? How? Why?
Student Essays of OU’s S288
S288-12B (2012 Course)
• Analysis of student essays through XIP
• Comparison of XIP findings with final grades
• To see whether XIP can identify important parts of student essays
• To see whether or not we can correlate XIP results with final grades
Student Essays of OU’s S288
S288-13B (2013 Course)
Same as two above • Use of Google Docs for
collaboratively written report where we back up the revision history & analyse through XIP
Same as two above • To see whether XIP can reveal
interesting predictive patterns about the quality of the end document and the final grade.
Student Essays of OU’s S288
S288-14B (2014 Course)
Same as above • Develop software with XIP
Visual Analytics Dashboard integrated
• Get users’ reactions
Same as above • Analyses student essays &
provide real-time analytics of students’ essays as a feedback to students.
Student Essays (soft domains)
N/A Same as above Same as above • Test the discipline independency
of XIP
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within
student essays?
RQ2: To what extent is there a relation between the
existences of various kinds of argumentative discourse
moves in student essays with final grades?
Data Collection & Analysis What? When? How? Why?
Student Essays of OU’s S288
S288-12B (2012 Course)
• Analysis of student essays through XIP
• Comparison of XIP findings with final grades
• To see whether XIP can identify important parts of student essays
• To see whether or not we can correlate XIP results with final grades
Student Essays of OU’s S288
S288-13B (2013 Course)
Same as two above • Use of Google Docs for
collaboratively written report where we back up the revision history & analyse through XIP
Same as two above • To see whether XIP can reveal
interesting predictive patterns about the quality of the end document and the final grade.
Student Essays of OU’s S288
S288-14B (2014 Course)
Same as above • Develop software with XIP
Visual Analytics Dashboard integrated
• Get users’ reactions
Same as above • Analyses student essays &
provide real-time analytics of students’ essays as a feedback to students.
Student Essays (soft domains)
N/A Same as above Same as above • Test the discipline independency
of XIP
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within
student essays?
RQ2: To what extent is there a relation between the
existences of various kinds of argumentative discourse
moves in student essays with final grades?
RQ3: To what extent automated metadiscourse
analysis of discipline-independent student essays can be used to
provide formative feedback?
Data Collection & Analysis What? When? How? Why?
Student Essays of OU’s S288
S288-12B (2012 Course)
• Analysis of student essays through XIP
• Comparison of XIP findings with final grades
• To see whether XIP can identify important parts of student essays
• To see whether or not we can correlate XIP results with final grades
Student Essays of OU’s S288
S288-13B (2013 Course)
Same as two above • Use of Google Docs for
collaboratively written report where we back up the revision history & analyse through XIP
Same as two above • To see whether XIP can reveal
interesting predictive patterns about the quality of the end document and the final grade.
Student Essays of OU’s S288
S288-14B (2014 Course)
Same as above • Develop software with XIP
Visual Analytics Dashboard integrated
• Get users’ reactions
Same as above • Analyses student essays &
provide real-time analytics of students’ essays as a feedback to students.
Student Essays (soft domains)
N/A Same as above Same as above • Test the discipline independency
of XIP
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
RQ1: How reliable and sufficient are the automated discourse analysis tools for finding good attributes of scholarly writing within
student essays?
RQ2: To what extent is there a relation between the
existences of various kinds of argumentative discourse
moves in student essays with final grades?
RQ3: To what extent automated metadiscourse
analysis of discipline-independent student essays can be used to
provide formative feedback?
How can we support students’ scholarly
writing skills to improve the quality
of their writing through automated
metadiscourse analysis?
Validation of XIP
XIP
Quality
Grades
Science
Social Sciences
Art
History
Marking Rubrics
Representations
Educators
Tutors
Students
10
/09
/20
13
, Tu
esd
ay, T
he
Op
en
Un
iver
sity
M
ini V
iva
Pre
sen
tati
on
