NAPP: Connecting mentors and students at Tecnico Lisboa

Pedro Miguel Branco Veiga

Thesis to obtain the Master of Science Degree in

Information Systems and Computer Engineering

Supervisors: Prof. Ana Isabel Baptista Moura SantosProf. Jose Alberto Rodrigues Pereira Sardinha

Examination Committee

Chairperson: Prof. Antonio Manuel Ferreira Rito da SilvaSupervisor: Prof. Ana Isabel Baptista Moura Santos

Members of the Committee: Prof. Nuno Joao Neves Mamede

October 2017

Acknowledgments

I would like to thank my parents for their friendship, encouragement and caring over all these years,

for always being there for me and without whom this project would not be possible. I would also like to

thank my grandparents, aunts, uncles and cousins for their understanding and support throughout all

these years.

I would like to thank my brother for all of his support. He will always be my example.

I would like to thank my girlfriend, for all the support that she gave me throughout the last years of

my academic life. Her undisputed love gave me the necessary strength to overcome every personal and

work-related setback that I have encountered.

To my second family, Tuna Mista do Instituto Superior Tecnico (TMIST), my sincere thanks for all the

nights of ”foly” that I had with you, much more will come. I will always remember the first notes of our

anthem with a feeling of nostalgia.

NAPP concept was firstly idealized by Carla Boura, current NAPE-TP coordinator. Her perseverance

and dedication was essential to the project.

To my advisors, Prof. Ana Moura Santos and Prof. Alberto Sardinha, my thanks for the excellent

support during NAPP’s development.

Abstract

In the past seven years, a successful first-year mentoring programme at Tecnico Lisboa’s Taguspark

campus promoted by Nucleo de Apoio ao Estudante - Taguspark (NAPE-TP) was brought into play.

Nevertheless, the relationship between mentors (mostly second-year students) and mentees (first-year

students) tends to weaken after the first academic weeks of the semester due to poorly defined commu-

nication channels. This problem can be addressed with the creation of a consistent and unique commu-

nication channel between all the parties involved in this programme. This work presents NAPP, a novel

mentoring software solution for first-year mentorship programmes, that enhances the communication

between mentors and mentees while providing study guidance tools for mentees. NAPP is composed

of two key components, a cross-platform mobile application and a web application that is used as a high

level performance analysis tool by the programme’s coordinator. These components were developed

using state of the art technologies like the Ionic Framework using AngularJS, and the NoSQL databases

CouchDB and PouchDB. The undergone usability test to the mobile application component revealed that

the developed software was simple and intuitive to use with a SUS score of 89.42 [1] (in a scale of 0 to

100), which demonstrates a high level of user experience.

Keywords

Mentoring Program; Student Support Systems; Mobile Application; NoSQL Databases

iii

Resumo

Em 2010, o programa de mentorado do Tecnico Lisboa (campus Alameda, 1996) foi importado para o

campus do Taguspark, onde sofreu algumas adaptacoes. Este programa revelou-se, ao longo destes

7 anos, um sucesso junto da comunidade academica com especial enfase para os alunos de 1º ano.

Apesar dos resultados muito positivos, o programa ainda esta longe de ter atingido o seu maximo po-

tencial. Faz-se sentir a necessidade de uma nova abordagem com uma vertente mais tecnologica –

diretamente relacionada com a comunicacao entre: alunos de 1º ano – mentorados, alunos de outros

anos – mentores, e entre estes e o responsavel do Nucleo de Apoio ao Estudante. Concretamente, sen-

timos a necessidade de desenvolver uma aplicacao movel que ira permitir otimizar a comunicacao entre

as partes envolvidas no processo, tendo sempre como objetivo final uma maior eficacia na prevencao

ao insucesso escolar realizada pelo programa de mentorado. Esta solucao e composta por dois com-

ponentes, uma aplicacao movel hıbrida e uma aplicacao web que sera utilizada pelo coordenador do

programa. Estas componentes foram desenvolvidas utilizando tecnologias como a Ionic Framework su-

portada por AngularJS, e as base de dados NoSQL CouchDB e PouchDB. O teste de usabilidade que

foi feito a componente da aplicacao movel revelou que o software desenvolvido e simples e intuitivo,

e este tem uma pontuacao SUS de 89.42 [1] (numa escala de 0 a 100) o que indica um alto nıvel de

experiencia de utilizacao.

Palavras Chave

Programa de Mentorando; Sistemas de Suporte aos Alunos; Aplicacao Movel; Base de Dados NoSQL

v

Contents

1 Introduction 1

1.1 NAPE-TP Mentoring Process and Problem Context . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Objectives and Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Document Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 State Of The Art 7

2.1 Student Supporting Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Chronus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.2 My Study Life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.1.3 Focus Booster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 Mentoring Programmes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.1 First-Year Undergraduate Students Mentoring Programmes . . . . . . . . . . . . . 11

2.2.2 Alumni Mentoring Programmes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.3 Other Mentoring Programmes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Mentoring Software Solution 17

3.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.1 Selecting the Right Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.1.2 NAPP’s Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Sources of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.4 NAPP mobile app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.5 NAPP web app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4 First Results 37

4.1 Evaluation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2 Usability Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5 Conclusion 43

5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

vii

viii

List of Figures

1.1 NAPE-TP mentoring program communication channels for academic integration . . . . . 4

1.2 NAPE-TP mentoring program communication channels for academic success . . . . . . . 5

2.1 Chronus workflow to create an impacting mentoring programme [2] . . . . . . . . . . . . . 9

2.2 Average of admission grades and the approved ECTS in the first year, according to the

academic year and campus [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3 The Effective Learning Framework (ELF) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.1 NAPP’s logo first version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.2 Developer’s awareness of cross-platform tools in 2014, according to Research2guidance [4] 22

3.3 NAPP’s architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.4 Apache Cordova architecture [5] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.5 Apache Cordova plugin architecture [5] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.6 Webxity, CouchDB-AngularJS offline syncing app - Use Case [6] . . . . . . . . . . . . . . 30

3.7 Login Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.8 Create Account Panel Alert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.9 Mentor’s Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.10 Mentor’s Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.11 Mentee’s Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.12 Mentee’s Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.13 Mentee’s To-do List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.14 Mentee’s Pomodoro Timer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.15 Web Application Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.16 Web Application Smart Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.1 Usability Test Results - Time to conclude each task . . . . . . . . . . . . . . . . . . . . . . 40

4.2 Usability Test Results - Error rate of each task . . . . . . . . . . . . . . . . . . . . . . . . . 41

ix

x

List of Tables

3.1 Mobile development approaches [7] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.2 Comparison between different frameworks [8] . . . . . . . . . . . . . . . . . . . . . . . . . 23

xi

xii

1Introduction

Contents

1.1 NAPE-TP Mentoring Process and Problem Context . . . . . . . . . . . . . . . . . . . 3

1.2 Objectives and Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Document Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1

2

Formerly known as Gabinete de Apoio ao Estudante (GAPE), Nucleo de Apoio ao Estudante (NAPE)1

was created on the 3rd of December of 1990 from the combined efforts of Instituto Superior Tecnico (IST

or Tecnico Lisboa)2 and Direcao-Geral do Ensino Superior (DGES)3 to improve the academic success

rates of Tecnico Lisboa. This initiative was inserted in the National Programme to Combat Failure at

School [9], promoted by the Portuguese Ministry of Education.

As the interface between students and the management offices of Tecnico Lisboa, NAPE’s mission

consisted from the very start of the following actions: promotion of the relationship between students and

the University, fostering of initiatives that help student’s integration, reinforcement of student’s activities,

creation of activities to welcome first-year students and publicize the Tecnico Lisboa’s graduate courses

offer to high school students.

NAPE’s main student support service is the mentoring programme. The main focus of this pro-

gramme is to welcome, integrate and assist students that are admitted for the first time in Tecnico

Lisboa, mainly first-year and international students, into academic life. With the help of NAPE’s mentors

and guides, mostly second/third year students, the newcomers get personalized assistance during their

first steps in Tecnico Lisboa’s academic life.

The mentoring programme started in 1996 at first to support and assist students of the Electric

and Computer Engineering bachelor degree. Years after, the programme was applied in 17 bachelor

degrees. Starting from the academic year of 2010/2011, the Mentoring Programme had a fresh start in

the Taguspark campus.

Even though the programme is well organized, there is a recognized communication problem be-

tween the three parties involved on it: NAPE-TP’s coordinator, mentors and mentees. Therefore, there

was a need for a software solution that supports the information flow between these parties and also

integrates study guidance tools that help students throughout their academic life.

1.1 NAPE-TP Mentoring Process and Problem Context

NAPE-TP mentoring program depends, in what concerns the general academic integration of first-

year students, on the exchange of information between the following parties (see Fig.1.1).

1https://nape.tecnico.ulisboa.pt/nape/2https://tecnico.ulisboa.pt/en/3http://www.dges.gov.pt/pt

3

Figure 1.1: NAPE-TP mentoring program communication channels for academic integration

I Invite and distribute mentors to all first-year students;

II Establish a relationship of trust during the enrolment week in Taguspark campus;

Give academic support and share university-related experiences;

Suggest the attendance of study halls and groups;

III Report problems related with the mentoring program.

The distribution referred in (I) is a manual process in which the NAPE-TP’s coordinator matches

every first-year student, usually 300 at Taguspark campus, with a mentor, from a pool of around 90

students.

Analysing channel (II), it is possible to conclude that, after many editions of the NAPE-TP mentoring

programme, a relationship of trust is not established during the enrolment week. After the first weeks

of the semester this situation results in the end of the relationship compromising the goals of the pro-

gramme.

Regarding (III), NAPE-TP has no direct feedback of the mentoring programme from mentees, only

from mentor’s reports.

Moreover, in what concerns the academic success of first-year students, the exchange of information

is done as indicated in the next diagram (see Fig.1.2).

4

Figure 1.2: NAPE-TP mentoring program communication channels for academic success

1. Psychological and/or academic personalized support;

Suggest a tailored academic plan in case of poor academic performance;

2. Report academic performance, mainly the first test grades;

3. Report mentee’s grades.

If a mentor reports a case of a mentee with poor academic performance (fail in three or more eval-

uations), the mentee will be invited to an interview with NAPE-TP’s coordinator in order to find a quick

solution. This solution can be a personalised academic plan for the next semester that doesn’t overload

the student, or psychological support from Tecnico Lisboa’s Psychological Assessment and Support

Services, for example.

Until now, the process of reporting grades to mentors is entirely dependent on the exchanging of e-

mails between the two parties that intervene in channel (2), with the exception of the cases where there

is a personal relationship. This communication process is usually very delayed, meaning that mentors

have to pressure their mentees in order to get their feedback.

The delay referred in (2) cascades to (3), leading to a desynchronised communication which results

in the overburden of the programme’s coordinator.

5

1.2 Objectives and Requirements

To fulfil the need of a framework that supports the referred information flow, a software solution

was developed to support both processes (see Fig.1.2 and Fig.1.1). The following requirements were

identified for this new mentoring support solution:

• A communication module between the three parties of the mentoring system: NAPE-TP’s coordi-

nator, mentors and mentees;

• A reporting module that receives as input mentee’s grades. This module would firstly be used by

mentors to efficiently follow their mentees academic track record evolution and, at a higher level,

by NAPE-TP’s coordinator to follow mentor’s work and effectiveness;

• A module with study guidance tools for first-year students;

• A feedback module to obtain regular and updated suggestions related to the programme from

mentors and mentees.

In this context, there was an opportunity and value in implementing an application framework to sup-

port the information flows and generate attraction to the programme by including also relevant guidance

content for mentees. For the mentoring programme’s coordinator, the added value provided by this ap-

plication framework would be the ability to monitor the mentees’ performance and the mentor activities

in real time. For mentors and mentees, the application framework would provide a mentoring workflow

that enables an optimization in information sharing as per the objectives mentioned above.

Regarding the published work, this application framework was featured in one international and one

national conference. Accepted as a poster paper, this solution was published in the proceedings of the

12th European Conference on Technology Enhanced Learning, EC-TEL 2017 [10], hosted in Tallinn

(Estonia). Also, this solution was presented in CNaPPES 17 [11], ”4º Congresso Nacional de Praticas

Pedagogicas no Ensino Superior”, a portuguese conference hosted in IPS (Polytechnic Institute of

Setubal, Portugal).

1.3 Document Outline

The document structure will be the following: in chapter 2 will be described the state of the art

student supporting software and mentoring programmes; in chapter 3 the mentoring software will be

detailed concerning its architecture and main components; in chapter 4 the results of the undergone

usability tests will be analysed; finally, in chapter 5, the document will end with the conclusion and the

identified future work.

6

2State Of The Art

Contents

2.1 Student Supporting Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 Mentoring Programmes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

7

8

2.1 Student Supporting Software

From the existing student supporting software solutions, the ones that influenced the developing

process of the referred mentoring framework application will be described in the following section.

2.1.1 Chronus

Across the globe, many mentoring programs, with different objectives (student onboarding or student-

alumni support, for example), rely on Chronus. This software is implemented in universities like the Sloan

School of Management of the Massachusetts Institute of Technology and the University of Washington.

It features a matching algorithm that takes into account the academic or professional profile of each

individual, a programme manager that automates mentoring tasks increasing their effectiveness and a

reporting tool to analyse the programme impact in the target organization [12].

Chronus has an iOS and Android mobile application that have all the features of the pre-existent web

application. This mobile application has increased user participation by 34% and engagement within

mentorships by 47%. With the usage of push notifications, users are frequently engaged within the

mentoring program, increasing their activity [13].

Although Chronus was initially a mentoring software for professional organizations, it is now a refer-

ence in modern mentoring within the context of universities because it drives strategic value. University

mentoring variations like career mentoring, peer mentoring, student onboarding and alumni mentoring

are all supported by Chronus. Figure 2.1 describes the Chronus workflow to create a mentoring pro-

gramme with impact.

Figure 2.1: Chronus workflow to create an impacting mentoring programme [2]

Concerning the connection phase between mentors and mentees, Chronus has a matching algorithm

that takes into account participants’ competencies, backgrounds, learning styles and needs. According

to Chronus guidelines to create a high-impact mentoring programme, in cases of large groups of mentors

and mentees, the best option is to use automatic bulk matching because usually it is impossible to do a

one-to-one personalized matching in a timely manner [2].

From the developed application framework contextual perspective, Chronus is an example of a robust

and complex mentoring software system specially in what concerns its matching algorithm. Although it

9

supports the type of mentoring approach that is applied in Tecnico Lisboa (to be described in section

2.2), Peer Mentoring 1 and Student Onboarding 2, it is a very costly solution. An in-house tailored

solution will be developed to fulfil the specific requirements detailed in section 1.2.

2.1.2 My Study Life

As one of the most popular study planner mobile applications in the main app stores, Android’s Play

Store, iOS’s App Store and Windows Phone’s Store, My Study Life helps students schedule in a week

or in a day-to-day scope. With the ability to create to-do lists and set reminders associated with tasks

and future exams, students can easily manage their academic responsibilities [14].

Aside from the mobile applications, My Study Life also has a web app that can be accessed through

any web browser and a Windows desktop application.

My Study Life software has two key features:

• Student’s information is stored in the cloud and syncs across all devices;

• It is possible to check timetables and add tasks in an offline enviroment. The sync cloud is done

as soon as the user is back online.

2.1.3 Focus Booster

Based on the Pomodoro Technique, Focus Booster is a web and desktop application designed to

provide a more in-depth experience of a simple Pomodoro timer. It features a dashboard with reports of

the user’s time efficiency based on a real-time Pomodoro analysis [15].

The Pomodoro Technique was developed by Francesco Cirillo, a CEO of XPLaps in Germany. It is

a management method which uses a fixed block of typically 25 minutes (called pomodoro sessions),

followed by a 5 minute break. Pomodoros cannot be interruped or split. The objective is to focus only

on a single given task during the 25 minutes, while fully relaxing in the 5 minute break. For every four

Pomodoros, the user is allowed to have a 15-30 minute break [16].

My Study Life and Focus Booster platforms include study guidance tools as to-do lists and time

management techniques that are commonly used by students. The features that were implemented in

NAPP with the purpose of increasing its value to first-year students were: a to-do list, which stores the

completed tasks and can be updated in both online and offline environments, and a pomodoro timer.

1http://chronus.com/software/mentoring-software/student-peer-mentoring2http://chronus.com/software/mentoring-software/new-student-onboarding

10

2.2 Mentoring Programmes

Mentoring Programmes are globally subdivided between two type of programmes: First-Year Under-

graduate Students Mentoring Programmes and Career Mentoring Programmes.

2.2.1 First-Year Undergraduate Students Mentoring Programmes

In this type of programmes mentors are mostly second/third-year volunteer students with good com-

munication and interpersonal skills who have a good academic track record. Mentors provide study

guidance and share their academic life experiences so that their mentees achieve the best possible

results for their work during their first university months [17]. Here are the main examples of universities

that have First-Year Undergraduate Students Mentoring Programmes:

• Instituto Superior Tecnico - Campus Taguspark

In Taguspark campus, mentors are called upon to participate on the grounds of their communica-

tion, interpersonal skills and academic performance on the previous semesters, by the programme

coordinators, Carla Boura from NAPE-TP and Prof. Ana Moura Santos.

Mentors are responsible for supporting the logistics of the enrolment week at Taguspark campus.

During this event a group of 5/6 first-year students is attributed to each mentor. The matching

process takes into account only the course of each student. There are four courses in this cam-

pus: Computer Science and Engineering (LEIC) 3, Telecommunications and Informatics Engineer-

ing (LETI) 4, Electronics Engineering (LEE) 5, Industrial Engineering and Management (LEGI) 6.

Across the years, LEIC has been the course that has had more students and LEGI is the course

with higher mentor/mentee ratio.

In the first semester, mentors are required to participate in the welcoming session of the first-

year students and establish the first contact with their mentees. During this semester, mentor’s

responsibilities are: establish a trustworthy relation with their mentees, follow their mentees’ first

grades, suggest the attendance at study halls and groups for first-year students and encourage

mentees to attend the office hours supported by the professors. Mentors also help to early track

on a bad performance of mentees. If for some reason, certain students had bad grades in their first

evaluations, they are invited to personal interviews (see also Fig. 1.2). These interviews have the

objective to define a personalized study plan so that those students actively engage in improving

their own academic performance.3https://fenix.tecnico.ulisboa.pt/cursos/leic-t4https://fenix.tecnico.ulisboa.pt/cursos/lerc5https://fenix.tecnico.ulisboa.pt/cursos/lee6https://fenix.tecnico.ulisboa.pt/cursos/legi

11

In the second semester, mentors are required to: continue their relationship of trust with their

mentees giving support relevant to their new challenges, reinforce the importance of the study

halls and aid their mentees with their test/exam preparation (see also Fig.1.1).

Mentors are encouraged to always promote, suggest and assist academic activities.

It is possible to see the positive impact of the NAPE-TP’s mentoring programme on the course of

Computer Science and Engineering of the Taguspark campus (LEIC, former LEIC-T) in Figure 2.2.

When comparing the results of first-year students who enrol in LEIC, at the Taguspark campus,

with students who enrol in the same course at the Alameda campus (LEIC-A), it is possible to

conclude that although the Taguspark students enter the University with lower admission grades,

they began to obtain better grades at the end of their first academic from 2011/2012 (see the

evolution of approved ECTS in Fig.2.2).

Figure 2.2: Average of admission grades and the approved ECTS in the first year, according to the academic yearand campus [3]

• Instituto Superior Tecnico - Campus Alameda

In Alameda campus, mentors are volunteers who are available to share their academic experience

and know-how with mentees. When enrolling, future mentors have to choose which category of

students they want to help, the categories being: first-year students, new master’s students and

international students. Mentors are managed by guides. Guides are students with scholarships

who receive an extra training in order to be responsible for the good performance of the mentors.

During the welcome week, each mentor is attributed to a group of twenty students to help them in

their enrolment in Tecnico Lisboa. From this moment on, these twenty students have an official

mentor that will help them throughout their first year in IST.

12

For the duration of the year, mentors have to assist their mentees with study materials and aca-

demic counselling. Also, mentees are engaged with three types of events: workshops, team

building and sharing sessions [18].

• TU Delft: Delft University of Technology

In TU Delft there are several faculties that promote mentoring programmes.

The faculty of Technology, Policy and Management has two different mentoring systems, one for

first-year students and another for Master programme Engineering and Policy Analysis (EPA) and

Master programme Management of Technology (MOT) students [19]. The first-year students of this

faculty are inserted in a group of around 10 students to receive advise concerning their course and

academic life. Both EPA and MOT master students are paired with a senior international student

and a senior Dutch student in order to exchange experiences [19].

Before the beginning of the semester, the Aerospace Engineering first-year students are invited to

the Freshmen Weekend. In this event, the new aerospace students get to know their colleagues

and can participate in several activities. The Reception Day, at the start of the academic year,

is where these students start to get acquainted to the academic life. During this day, they are

introduced to their own mentor and mentor group, and buy their faculty books and drawing materi-

als [20].

2.2.2 Alumni Mentoring Programmes

Alumni and finalists are paired in order to create a mutually rewarding relationship. In this case,

mentors provide mentees job search, career path, career interests and areas of study advise/support.

Here are the main examples of universities that have Career Mentoring Programmes:

• Carnegie Mellon University’s Heinz College

In this college, mentors are matched with mentees according to their career path, career interests

and area of activity. This matching is made through a LinkedIn group that automatically pairs

students and alumni who have similar profiles.

Mentors have the opportunity to give back to Heinz College by providing professional guidance to

mentees.

13

During this programme, mentees have the opportunity to broaden their contact network in the in-

dustry they are more passionate about [21].

• Massachusetts Institute of Technology (MIT) - Sloan School of Management

The mentoring programme in MIT Sloan is based in the relationship between students and alumni.

This programme is powered by Chronus (see description at section 2.1), a software to support

mentoring, to pair students and alumni according to their education, professional experience and

accomplishments, career, personal goals and interests. This information can be uploaded from

LinkedIn. The matching algorithm of this platform suggests to mentees multiple mentors that can

be paired with them.

Mentors should commit to a minimum of six months to one year to the programme in order to

form rewarding, mutually beneficial relationships with their mentees. Contrary to other mentoring

programmes, alumni aren’t encouraged to provide job search advise [22].

2.2.3 Other Mentoring Programmes

• KTH Royal Institute of Technology

KTH mentoring programme is designed so that students develop their employability skills and

develop their own self efficacy based on the Effective Learning Framework (ELF), see Fig.2.3.

The programme has a duration of 12 calendar months. In this period mentors are encouraged to

share their materials and literature with their mentees. Also, closed follow up meetings should be

scheduled between mentors and mentees at least four times during the year [23].

The International Buddy Programme is a KTH initiative that will be launched in the autumn of

2016. The pilot phase will focus KTH students travelling on exchange studies into Singapore,

Switzerland, Spain and the United States.

The mentors of this programme are alumni that are stationed in the areas that current KTH stu-

dents (mentees) will be studying. As global ambassadors of KTH, mentors have the opportunity to

make a positive impact on the mentees’ international experience by sharing local knowledge and

support. Mentees will receive from mentors personal and professional advices in order to increase

their carrer prospects.

14

Figure 2.3: The Effective Learning Framework (ELF)

• University of Michigan

The first-year students of the University of Michigan have the possibility to apply to a mentoring

programme since 1991 [24]. The application period is in April and over the summer they are

matched with a peer mentor (second-year students, at least) and a faculty/staff mentor. Peer

mentors are responsible for establishing a relationship with each of his 5/6 mentees so that they

know someone as they come to campus. Each mentoring group is engaged with team building

activities, leadership and soft-skills workshops and other campus-driven events.

The mentoring programme starts its activity with the Mentoring Kickoff, a networking annual event

that unites all the mentees, peer mentors and staff/faculty mentors [24].

15

16

3Mentoring Software Solution

Contents

3.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Sources of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.4 NAPP mobile app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.5 NAPP web app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

17

18

As it was described in section 1.1, the main problem faced by NAPE-TP’s mentoring programme is an

undefined and inefficient communication channel used in both processes (see Fig.1.2 and Fig.1.1). Al-

though there was an attempt that the exchange of information was done through emails, the programme

coordinator always resorted to other methods to reach mentors and mentees to avoid the inefficiency

of emails. There was some not very successful efforts to overcome this situation using Facebook and

Slack, but after some time the students failed to regularly use these platforms.

The proposed solution is subdivided in two components. The first is a mobile application, for mentors

and mentees, that is developed to increase their engagement in NAPE-TP’s mentoring programme.

This app provides mentors with academic performance tracking reports of their mentees and key study

guidance tools. The second component is a web application, developed for NAPE-TP’s coordinator, that

enables a high level view of the mentees’ academic performance evolution and the mentoring activities

carried out by mentors.

The communication, study guidance and feedback modules, referred in section 1.2, are exclusively

supported by the mobile application. The part of the reporting module, also described in this section, re-

lated to the mentees’ academic track record analysis is implemented on the mobile application. The part

referring to NAPE-TP’s coordinator high level monitoring is the key functionality of the web application.

The chosen name for this mobile application was NAPP, which is the result of the word binding

between NAPE (Nucleo de Apoio ao Estudante) and App (Mobile Application). As can be viewed in

Figure 3.1, the final version of the logo of NAPP was created using vector graphics. The objective

was to reflect the main target audience of the app, students, while using Tecnico Lisboa’s official color

palette [25].

Figure 3.1: NAPP’s logo first version

3.1 Approach

With the growing popularity of mobile devices as smart phones and tablets, the process of choosing

the right mobility solution has become more complex than ever. It is true that user engagement is higher

when the point of contact is through mobile devices, but this type of systems come with a great number

of challenges. A wide variety of screen sizes and capabilities make mobile development an intricate

process.

19

With the presently existing technologies, mobile development can be approached in three different

ways:

• Native Apps

Applications developed using native code continue to be the ones which provide the best user ex-

perience. Due to their optimization aimed to a particular operation system, they can efficiently use

device hardware (camera, for example) and software (calendar, for example). Native Apps can run

offline because they are installed locally on each device. In the native environment, applications

have full access to the device features, whereas multi-platform applications can only have access

to certain features by the use of libraries that can compromise the overall performance.

• Cross-Platform Apps

Cross-Platform Frameworks like PhoneGap 1, for example, enable a more cost and time efficient

mobile development because developers only design and produce a given application once and

then adapt OS-dependent modules, using the framework, to the multiple platforms. Web devel-

opers don’t have to learn any native programming language because these frameworks use web-

base technologies like Javascript and HTML.

• Mobile Web Apps

Any smart phone or tablet equiped with a web browser can run a Mobile Web App. The possible

audience of a web app is much broader than a native or cross-platform app because, when com-

pared with the last ones, it is device and platform independent. The limitation of these types of

apps is their performance and internet dependability, with the exception of the where a Progressive

Web App (PWA) [26] is implemented. These type of apps provide near-native user experience but

rely on the cached resources to remove the dependence on the network. However, web apps,

in general, have shown to be easy and efficient to produce and, nowadays, there are almost no

restrictions in the interaction between HTML5 and the device’s hardware with the use of features

like location tracking and integrated multimedia [27].

The referred approaches are summarized and compared according to four characteristics in Table

3.1:1http://docs.phonegap.com/

20

Table 3.1: Mobile development approaches [7]

Development Ap-proach

Native Cross-Platform Mobile Web

Definition andTools

Build the appusing nativeframeworks:iPhone SDKAndroid SDKWindows Phone SDK

Build once, deploy onmultiple platforms asnative apps:Ionic FrameworkPhoneGapTitanium Appcelerator

Build using webtechnologies:HTML5SenchajQuery Mobile

Underlying Tech-nology

iOS: Objective C andSwiftAndroid: JavaWindows Phone: .NET

Ionic: AngularJS, Javascript,HTMLPhoneGap: Javascript,HTMLAppcelerator: Javascript,HTML

Javascript, HTML

Deployment App Stores App Stores Over the web

Key Use CasesApps requiring extensivedevice and/or OSfunctions

Multiple device typesdistributed across key usersSimpler apps, moreinformational in nature

Generic user experience,performance depends onInternet connection

These three paths of mobile development are shortly described in the table in order to enable a com-

parison between them according to cost of development, multi-platform compatibility, user experience,

the provided features and overall performance.

In what concerns the cost of development of a mobile app, the cross-platform frameworks and mo-

bile apps have an advantage, because these applications only have to be developed once for multiple

platforms. Native apps have to be re-built for each native operating system which results in an excessive

developing budget comparing to the other approaches.

Analysing multi-platform compatibility, web apps have the advantage of achieving the wider reach

because they can be used with any mobile browser. Cross-platform apps typically can reach all main

native operation systems with few adjustments. Native apps just run on the mobile operation system

that they were directed for.

The best user experience is provided by native apps. Cross-platform apps can have the look and feel

of a native app, but in some cases they lack touch responsiveness. In web apps, user experience can be

affected due to differences between web browsers which produce small changes in the user interface in

each mobile operation system. This small differences can create several platform-specific bugs resulting

in the hindering of user experience [28].

In the matter of features, native apps have access to all of the device features and have the best

performance. Cross-platform and web apps can have access to all device features though plugins and

libraries. Cross-platform apps outperform web apps hosted in browsers because they don’t depend

on the internet connection. While comparing to Progressive Web Apps [26], cross-platform apps have

similar performance.

21

Figure 3.2: Developer’s awareness of cross-platform tools in 2014, according to Research2guidance [4]

3.1.1 Selecting the Right Approach

NAPP was developed as a cross-platform mobile application. The key aspects that influenced this

decision were multi-platform compatibility and capacity to fully run offline. Only cross-platform apps can

fully offer the combination of these two features.

Multi-platform compatibility was a priority in the development of this app. All platforms are equally

important because all the students involved in NAPE-TP’s mentoring program must be able to access

the app through their devices.

Users shouldn’t need an internet connection to access the information saved in NAPP. In an offline

environment this information could be possible desynchronized but in due course the app will merge its

content with the one that is in the remote server.

Also, nowadays, with the available frameworks, the user experience of a cross-platform app is nearly

as sleek as a native one. With the planned functionalities described in section 1.2, users didn’t feel any

disturbance in the performance of the app as was analysed in section 4.2.

With the development of HTML5 technology, cross-platform mobile application development tools

such as PhoneGap, jQuery Mobile 2, Adobe Air 3 and Titanium 4 have emerged to be the solution

when developers need to reduce cost of development and broaden the covered platforms that users are

currently using [29]. The developer’s awareness of these tools can be seen in Figure 3.2.

PhoneGap is an open source framework based on Apache Cordova that enables developers to

use web technologies (HTML, CSS and JavaScript) to build applications with native functionality. By

2https://jquerymobile.com/3http://www.adobe.com/pt/products/air.html4https://www.appcelerator.com/Titanium/

22

providing access to all native APIs, such as camera and GPS, this framework uses native code to

interact directly with the operating system.

While PhoneGap is responsible for packaging up web applications into multi-platform native apps,

Ionic Framework focus on delivering the best user experience possible using mobile optimized HTML,

CSS and JavaScript components, gestures, and tools to build highly interactive applications.

Ionic Framework essentially documents and promotes recommended design patterns which it pro-

vides itself, simplifying the complex development of the UI of mobile applications making use of Angu-

larJS.

AngularJS in turn is a strutctural framework for dynamic web application development. Using HTML

as template language, Angular objective aims to extend HTML’s syntax to express clearly and succinctly

the components of an application.

Analysing Table 3.2 it is possible to conclude that Ionic Framework is the best option because it

supports both Cordova and AngularJS and has a high native-like performance of 70%. These charac-

teristics make Ionic the best cross-platform mobile development framework, and that is why it was the

natural choice to build up NAPP mobile application.

Table 3.2: Comparison between different frameworks [8]

Framework Native-LikePerformance

SupportCordova/AngularJS

Ionic Framework 7/10 Yes/YesOnsen UI 6/10 No/YesFramework 7 8/10 No/NoReact Native 8/10 No/NojQuery Mobile 3/10 No/NoNative Script 8/10 No/No

3.1.2 NAPP’s Database

The database technologies that were implemented in NAPP were CouchDB 5 and PouchDB 6, both

of which are NoSQL databases.

NoSQL databases use schemaless data, run on clusters, and have the ability to trade-off traditional

consistency for better performance and scalability. Moreover, these databases are also easier to pro-

gram with [30]. One key aspect that differentiates NoSQL from SQL databases is that these last ones,

besides the usage of the same common programming language, SQL, they also always use the concept

of tables.

NAPP’s database development approach was chosen based on the following key features: NoSQL

systems are designed to work with large volumes of information that are analysed in real time [31]. Also,5http://docs.couchdb.org/en/2.1.0/6https://pouchdb.com/api.html

23

this type of databases were created mostly for dealing with highly distributed databases [32].

There are four main types of NoSQL databases:

• In-memory key-value stores - every item in the database is stored as an attribute name (or ”key”)

together with its value in a schema-less way. The most popular in-memory key-value databases

are Redis 7, VoltDB 8 and Memcached 9;

• Column stores - data is stored as columns instead of rows, excelling in operations over large

datasets. The most popular column store databases are Apache HBase 10, Cassandra 11 and

Google’s BigTable 12;

• Document databases - store records as documents that contain many different attributes in the

form of key-value pair, key-array pair or nested documents. The most popular document-oriented

databases are CouchDB and MongoDB;

• Graph databases - stores information about networks excelling at dealing with interconnected data.

The most popular graph databases are InfiniteGraph 13, Neo4J 14 and OrientDB 15.

In summary, CouchDB and PouchDB, while NoSQL databases, were the right choices for NAPP

because they are best suited for managing data entities with multiple attributes with different types, such

as people, and applications that store and exchange information in the form of JSON data structures,

yet to be described in section 3.2.

CouchDB is an open-source project, written in Erlang, based on the document-oriented approach. It

was designed to run in distributed systems using the Couch Replication Protocol [33].

PouchDB is based on CouchDB, and constitutes an in-browser database that allows applications

to save data locally, so that users can experience all the features of a certain mobile application when

they are offline. Written in Javascript and running in Node.js, it can be used as a direct interface to

CouchDB-compatible servers.

Both previously described databases run in Cordova/PhoneGapp apps and can be used with Angu-

larJS. This creates a full-stack ecosystem composed by the Ionic Framework, PouchDB and CouchDB.

The data entities that are manipulated by this application framework are students. Each student,

either mentor or mentee, has a JSON document in NAPP’s database with their personal academic

information (their istID, for example) and other information related to their NAPP mobile application

7https://redis.io/8https://www.voltdb.com/9https://memcached.org/

10https://hbase.apache.org/11http://cassandra.apache.org/12https://cloud.google.com/bigtable/13http://www.objectivity.com/products/infinitegraph/14https://neo4j.com/15http://orientdb.com/orientdb/

24

account (the derived form 16 of their password, for example). Each of these documents are present both

in the student’s local PouchDB mobile database and in the single CouchDB server database, and are

synchronized in a bidirectional way in real-time.

Concerning both mentors’ and mentees’ JSON documents, there are common fields that can be

seen in the following JSON object:

1 {

2 " id": "ist171111",

3 " rev": "51-baf94df8b4df4fb89a1763da46144ddf",

4 "firstName": "Luis",

5 "lastName": "Santos",

6 "phoneNumb": "99991234",

7 "locality": "Oeiras",

8 "email": "luis.santos@tecnico.ulisboa.pt",

9 "course": "LEIC",

10 "istID": "ist171111",

11 "password": {

12 "type": "Buffer",

13 "data": [

14 10,

15 88,

16 49,

17 221,

18 64,

19 55,

20 236,

21 (...)

22 ]

23 },

24 "salt": "8317871b9e7634ae7d29af3397a2ddc2",

25 "studentType": "mentee"

26 }

The following list describes the non student type specific fields:

• id - this is the document’s primary identifier in the database. This field can be a auto-generated

string by PouchDB or CouchDB, or set by the programmer to a specific string. In NAPP’s context,

the id is the student’s IST ID;16https://en.wikipedia.org/wiki/Key derivation function

25

• rev - with this auto-generated revision string, PouchDB and CouchDB can check the document’s

version when it is fetched, updated or removed;

• firstName - student’s first name;

• lastName - student’s last name;

• phoneNumb - student’s phone number;

• locality - the locality where the student lives;

• email - student’s email;

• course - student’s course;

• istID - student’s IST official identifier;

• password - student’s password after it is derived by the Password-Based Key Derivation Function

2 (PBKDF2) algorithm described in [34];

• salt - the salt to be used in the referred PBKDF2 algorithm. This string is saved in each student’s

document in order to authenticate them in the login process through NAPP’s mobile app;

• studentType - student’s type, either mentor or mentee;

If the student is a mentee the following fields will be present in their JSON document:

1 {

2 "mentor": "ist176364",

3 "checkmarks": [

4 false,

5 false,

6 false,

7 false

8 ],

9 "positive": 2,

10 "negative": 0,

11 "mean": 13.15,

12 "grades": [

13 {

14 "grade": "14.3",

15 "course": " Algebra Linear "

16 },

26

17 {

18 "grade": "12",

19 "course": " Introducao a Engenharia Informatica "

20 }

21 ],

22 "toDoTasks": [

23 "Reunir com mentor",

24 "[Mentor] Fazer ficha de CDI 1!"

25 ],

26 "doneTasks": [

27 "Organizar estudo para Algebra Linear"

28 ],

29 "numDoneTasks": 1,

30 "numCompletedPomodoros": 2

31 }

The following list describes the mentee’s specific fields:

• mentor - the IST identifier of the mentee’s mentor;

• checkmarks - the current state of the pomodoro’s cycle. The true/false values indicate the number

of working time intervals that were already concluded;

• positive - student’s number of passing grades;

• negative - student’s number of failing grades;

• mean - student’s academic mean concerning his reported grades;

• grades - list of the student’s reported grades;

• toDoTasks - student’s list of the to be completed tasks;

• doneTasks - student’s list of completed tasks;

• numDoneTasks - student’s total number of completed tasks;

• numCompletedPomodoros - student’s total number of completed pomodoro cycles.

If the student is a mentor the following fields will be present in their JSON document:

1 {

2 "mentees": [

27

3 "ist171111",

4 "ist172222",

5 "ist173333",

6 "ist174444"

7 ],

8 "menteesFeed": [

9 "Miguel Marques obteve 15.6 a Algebra Linear .",

10 "Joana Silva obteve 12 a Sistemas Digitais .",

11 "Luis Santos obteve 14.3 a Algebra Linear "

12 ],

13 "numAssignedTasks": 4

14 }

The following list describes the mentor’s specific fields:

• mentees - list of IST identifiers corresponding to the mentees matched to the mentor in question;

• menteesFeed - list of messages corresponding to the last activities performed by the mentees;

• numAssignedTasks - total number of tasks that were assigned to the mentees.

3.2 Architecture

NAPP’s architecture is based mainly on three technologies: Ionic Framework, PouchDB and CouchDB

(see Fig.3.3).

On the client side (mentor’s and mentee’s side), the multiple NAPP mobile applications are able to

keep their local PouchDB database up-to-date even when the users are offline. On the server side

(NAPE-TP’s side), NAPP web application provides access to the information on the CouchDB server

that is synchronized with all PouchDB instances.

The Ionic Framework leaves app bundling to Cordova. Apache Cordova is a platform for web-based

hybrid apps and its high level architecture is schematically shown in Figure 3.4.

Cordova supports and interacts with many different device-specific APIs and combines all the APIs

from a native environment in one JavaScript API that is accessible by the hybrid app. These apps run

inside a WebView under the control of Cordova Framework.

Aside from this, Cordova communicates through an encapsulated channel with native device APIs

with the usage of plugins. These plugins obtain basic device information and device’s geolocation, for

example.

Cordova’s plugins are subdivided in two parts. The first part runs in JavaScript within the WebView

in order to provide an API to the hybrid app. The second part is written in the device’s platform native

28

Figure 3.3: NAPP’s architecture

Figure 3.4: Apache Cordova architecture [5]

code with the purpose of controlling the native APIs. Figure 3.5 schematically describes the high level

plugin architecture.

CouchDB has two unique features: HTTP and Sync. Its primary means of communication is HTTP,

therefore there is no need for a intermediary layer that converts database queries into RESTful HTTP

calls. Being CouchDB based only on REST and HTTP, it is possible to communicate with it using only a

web browser. CouchDB was built from origin to be capable of supporting synchronization. It is possible

29

Figure 3.5: Apache Cordova plugin architecture [5]

to establish two-way replication between CouchDB server instances. This means that users can speak

with different CouchDB servers across the globe without being concerned about latency in their client-

side applications.

PouchDB concept is to have the database inside the browser. Inside the Node.js enviromment,

PouchDB uses LevelDB and other backends via the LevelUP ecosystem 17.

An example of the data workflow between the PouchDB (hosted in the user’s app) and CouchDB

(server), can be seen in Figure 3.6.

Figure 3.6: Webxity, CouchDB-AngularJS offline syncing app - Use Case [6]

17https://github.com/Level/levelup/wiki/Modules

30

3.3 Sources of Information

Regarding the sources of information needed to develop the app, instead of the use of FenixEdu

Open API [35], mentors and mentees have to create an account using the NAPP mobile application

describing their personal information (name and course, for example).

Also, as expected, mentees have to manually input their grades through the mobile application. Any

missing or relevant data can be inserted in the NAPP web application by NAPE-TP’s coordinator.

3.4 NAPP mobile app

The first page of the NAPP mobile app is a login screen (see Fig. 3.7). In order to login, the user has

first to create an account. The alert that can be seen in Fig. 3.8, states that if the user of the app is a

mentee, he needs to be added by his mentor to his mentees list through the app. Only then the account

creation panel will be prompted with the corresponding mentee’s IST ID after he tries to login.

Figure 3.7: Login Screen Figure 3.8: Create Account Panel Alert

NAPP’s mobile application component was developed to be used by two different groups of students,

mentors and mentees. The mentor’s application viewpoint is composed of the following functionalities

(see Fig. 3.9 and 3.10): Mentor Dashboard (”Dashboard”) and List of mentee’s academic performance

analysis panels (one for each mentee).

31

With the Mentor’s Dashboard each mentor is able to see their mentees ranked by their academic av-

erage performance and an activity feed with their last updates. Concerning the JSON document’s fields

described in section 3.1.2, the referred academic average performance is obtained from the each of the

mentee’s ”mean” field and the referred activity feed is obtained from the logged mentor’s ”menteesFeed”

field. For example, this activity feed will inform mentors about their mentees last reported grades. The

mentor’s component of the reporting module described in section 1.2 can be accessed through each of

their mentees panels.

Figure 3.9: Mentor’s Dashboard Figure 3.10: Mentor’s Menu

The mentee’s application viewpoint is composed of the following functionalities (see Fig. 3.11 and

3.12): Mentee Dashboard (”Dashboard”), My Mentor (”O Meu Mentor”), To-do List (”Lista de Tarefas”),

Pomodoro Timer (”Temporizador Pomodoro”) and Mentoring Program Feedback (”Sugestoes”).

The Mentee Dashboard shows for each mentee his/her own overall average (mentee’s ”mean” field

described in section 3.1.2) along with the number of his/her passing and failing grades (mentee’s ”posi-

tive” and ”negative” fields described in section 3.1.2). Here the mentee can report a grade directly to his

mentor through the button Report Grade to Mentor (”Reportar Nota a Mentor”), adding an entry to the

mentee’s ”grades” field described in section 3.1.2.

The general information about the mentee’s mentor is described in the My Mentor functionality. This

information in obtained from the mentor’s ”firstName”, ”lastName”, ”phoneNumb”, ”locality”, ”email” and

”course” fields described in section 3.1.2.

The To-do List and the Pomodoro Timer functionalities are the study guidance tools referred in 1.1

section (see Fig. 3.13 and 3.14). The To-do List items that will be completed are stored in the mentee’s

32

Figure 3.11: Mentee’s Dashboard Figure 3.12: Mentee’s Menu

”toDoTasks” field and the completed ones are stored in the ”doneTasks” field, described in section 3.1.2.

When a certain task is completed, the mentee’s field ”numDoneTasks” is incremented by 1.

With the To-do List functionality mentees can organize their study process managing tasks by creat-

ing and completing them. To improve their productivity, the Pomodoro Timer functionality implements a

fully functional pomodoro timer that enables mentees to start/stop and reset a timer with 4 cycles of 25

minutes.

The feedback module is implemented in the Mentoring Program Feedback functionality. With this

functionality mentees can send a direct text message to the mentoring program coordinator.

33

Figure 3.13: Mentee’s To-do List Figure 3.14: Mentee’s Pomodoro Timer

34

3.5 NAPP web app

The NAPP web app was developed to be used by NAPE-TP’s coordinator. This web has two mod-

ules: a dashboard, with the most relevant information about the mentoring activity; a smart table, which

enables a profound and detailed analysis of the student’s data (mentors and mentees).

The dashboard (see Fig. 3.15) has: the number of mentors (”Mentores”) and mentees (”Mentoran-

dos”) in the mentoring program, the number of passing (”Notas Positivas”) and failing (”Notas Negativas”)

grades, the number of reported mentee’s grades (”Notas Reportadas”), the number of completed tasks

by mentees (”Tarefas Completadas”), the number of tasks that mentors assigned to their mentees (”Tare-

fas Atribuıdas”) and the number of full pomodoro cycles done by mentees (”Pomodoros Completados”).

All of these information is obtained from the CouchDB server database referred in section 3.2 and

described in section 3.1.2.

Figure 3.15: Web Application Dashboard

The smart table, or ”Tabela Dinamica” (see Fig. 3.16), is a tool to manipulate the student’s data

and sorting it by a certain field. Through this table it is possible to analyse, for example, the mentees

of a certain mentor by sorting the ”mentor” field. Also, here the mentoring program coordinator can

change directly the student’s data if, for example, the student wrongly inserted some information. The

changed information in this table is reflected on the single CouchDB server referred in section 3.2 and

consequently on the multiple NAPP mobile applications.

35

Figure 3.16: Web Application Smart Table

36

4First Results

Contents

4.1 Evaluation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2 Usability Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

37

38

4.1 Evaluation Plan

NAPP was tested according to usability. For all types of different software, usability tests consider

the following aspects [36]:

• More efficient to use (Effectiveness) - takes less time to complete a particular task;

• Easier to learn (Efficiency) - operations can be learned and completed consuming low resources;

• More user satisfaction (Satisfaction) - meets user expectations.

The existent types of evaluation methodologies [36] that are usually used to evaluate mobile usability

are:

• Laboratory experiments - human participants are required to perform specific tasks using a mobile

app in a controlled setting;

• Field studies - users are provided with mobile apps and asked about their experience;

• Hands-on measurements - defined aspects of mobile apps are measured directly.

In the case of the developed mobile application, the main part of NAPP framework, the field study

methodology was the most appropriate methodology to evaluate the app because, not only it could be

applied as very useful approach early in product development, but also allowed to gather direct user

feedback. In general, during field studies, data is collected about task flows, inefficiencies (errors during

tasks), and the organizational and physical environments of users.

4.2 Usability Test Results

A controlled group of 30 students (including mentors and mentees) undergone a usability test using

NAPP in a test environment (web browser) as soon as the prototype was created. The objective of

this first test was to evaluate mobile device-based gestures, content, interfaces, and the general user

experience [37].

The test was subdivided into two sets of tasks. The first set of tasks was with the mentor viewpoint

and the second was with the mentee viewpoint of the prototype app.

The mentor tasks were:

• Check a certain grade of a mentee in a specific course;

• Assign a task to a mentee;

• Order mentees according to a certain criteria (best grade).

39

And the mentee tasks were:

• Report a grade to the mentor;

• Add a task to the to-do list;

• Start a Pomodoro cycle;

• Send feedback of the mentoring program to NAPE-TP.

During each task, the test process was: assign the test task to the students, record the time that

was necessary to conclude the task, record the number of clicks that were necessary to finish it and

consequently the number of errors while doing it.

Measuring the effectiveness (referred in section 4.1) of the prototype app, the obtained results can

be seen in Figure 4.1

Figure 4.1: Usability Test Results - Time to conclude each task

With a mean of 9.41 seconds all tasks were rapidly completed.

40

The efficiency of the prototype app (referred in section 4.1) was measured based on the number of

errors while performing each task. The obtained error rate (at least one error during the task) can be

seen in Figure 4.2.

Figure 4.2: Usability Test Results - Error rate of each task

The maximum number of errors per task was 1 and the mean error rate was 10.4%.

With the above results it is possible to conclude that the app was successfully designed to be effective

and efficient because the test tasks were completed in an very short period of time and almost no error

was made.

After the tasks were concluded, the students had to fill in a short 10 question survey, based on Likert

scales 1, which was done according to the System Usability Scale (SUS) [1].

SUS is one of the most efficient and cost-friendly way of gathering statistically valid data and giving

a certain system, in this case an app, a precise usability score.

The 10 asked questions were:

• I think that I would like to use this system frequently.

• I found the system unnecessarily complex.

• I thought the system was easy to use.

• I think that I would need the support of a technical person to be able to use this system.

• I found the various functions in this system were well integrated.

1https://www.en.wikipedia.org/wiki/Likert scale

41

• I thought there was too much inconsistency in this system.

• I would imagine that most people would learn to use this system very quickly.

• I found the system very cumbersome to use.

• I felt very confident using the system.

• I needed to learn a lot of things before I could get going with this system.

Each of the questions had to be ranked from 1 to 5 based on how the tester student agreed with the

statement that they read. 1 means they strongly disagree and 5 mean they strongly agree.

To calculate the SUS score of each of the students the following mathematical operations were

made: For each of the odd numbered questions, it was subtracted 1 from the score; For each of the

even numbered questions, it was subtracted their value from 5; These new values added up are the total

score; The final score was obtained by multiplying the total score by 2.5 [38].

The mean SUS score of the tester students was 89.42, which is a very positive result that proves the

satisfaction (referred in section 4.1) level of the users while using NAPP’s prototype app.

42

5Conclusion

Contents

5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

43

44

5.1 Conclusion

Even though NAPE-TP mentoring programme can be considered a successful initiative from the

pedagogic point of view, its communication channels, as described in section 1.1, are undefined and

inefficient resulting in a desynchronized information flow. The developed mentoring framework, NAPP,

aims to solve this problem with an app developed thinking of first-year students that supports all the

required exchange of information, as referred in section 1.2, while providing study guidance tools. Be-

sides, the mentoring programme coordinator has access to a web application, integrated within NAPP

framework, with the relevant academic performance information of all the mentees.

Therefore, this framework is ready to modernize the mentoring programme, reducing the number of

manual processes that are still part of it, while increasing its impact and fostering the engagement of

first-years students through a mobile approach.

5.2 Future Work

NAPP next development phases should focus on the publishing process of the application in mobile

application markets (iOS and Android). After the publishing process is complete, a performance test

in real devices should be carried out to check first, if the defined solution architecture can support the

necessary exchange of information, and second, if the communication between the parties referred in

chapter 1 (NAPE-TP’s coordinator, mentors and mentees) was improved. To measure the performance

increase in all the involved communication processes the following key performance indicators (KPIs)

and metrics can be used: satisfaction of mentees with the feedback/support from mentors through the

mobile application (>60%), percentage of mentees that recognize the mentoring process was helpful

(>80%), satisfaction level of mentees with study guidance tools (>60%) and satisfaction level of mentors

and mentees with the feedback module (>60%).

During the usability test, several suggestions were made by mentors and mentees which shall be

taken into account in future implementations. The most demanded features were: access to a full

record of the achieved grades, while in the mentee’s perspective; possibility to create hypothetical grade

scenarios that lead to a certain final grade in a course, also in the mentee’s perspective; and finally, the

integration of NAPP in the Tecnico Lisboa’s FenixEdu 1 software ecosystem, which would allow the use

of its Centralized Authentication System (CAS) 2 and its API 3 with all the student’s information.

1http://fenixedu.org/2https://dsi.tecnico.ulisboa.pt/en/servicos/autenticacao-e-acesso/autenticacao-cas/3http://fenixedu.org/dev/api/

45

46

Bibliography

[1] J. Brooke et al., “Sus-a quick and dirty usability scale,” Usability evaluation in industry, vol. 189, no.

194, pp. 4–7, 1996.

[2] How to Start a High-Impact Mentoring Program, Accessed November 24, 2016, http://chronus.com/

how-to-start-a-mentoring-program.

[3] C. P. Filipa David, Ana Torres, Desempenho academico no campus do Taguspark: impacto do

projeto “Sucesso Escolar (NAPE-TP)” nos resultados do 1º ano, Feb 2014, accessed December 7,

2016 http://nep.tecnico.ulisboa.pt/download/Salas de estudo 201213 v24 02 14.pdf.

[4] R. . Guidance, Cross-Platform Tool Benchmarking 2014, 2014, accessed December 23, 2016 http:

//www.research2guidance.com/r2g/Cross-Platform-Tool-Benchmarking-2014.pdf.

[5] B. Ripkens, Ionic: An AngularJS based framework on the rise, 2014, accessed December 5, 2016

https://blog.codecentric.de/en/2014/11/ionic-angularjs-framework-on-the-rise.

[6] Webxity, CouchDB-AngularJS offline syncing app - Use Case, Accessed December 20, 2016, http:

//webxity.com/portfolios/couchdb-angularjs-offline-syncing-app/.

[7] How to Choose the Righ Architecture For Your Mobile Application, RapidValue Solutions, Nov 2012.

[8] S. Ottka, “Comparison of mobile application development tools for multi-platform industrial

applications; vertailu mobiilisovelluskehityksen tyokaluista teollisuuskayttoon tarkoitetuissa

monialustaisissa sovelluksissa,” G2 Pro gradu, diplomityo, 2015-05-11. [Online]. Available:

http://urn.fi/URN:NBN:fi:aalto-201505132661

[9] A. Benavente, Insucesso escolar no contexto portugues — abordagens, concepcoes

e polıticas., 1990, accessed December 22, 2016 http://analisesocial.ics.ul.pt/documentos/

1223034893G8cGD1nd2Zt45QL6.pdf.

[10] P. Veiga, A. Sardinha, A. M. Santos, and C. Boura, “Napp: Connecting mentors and students at

tecnico lisboa,” in Data Driven Approaches in Advanced Learning, 12th European Conference on

Technology Enhanced Learning (EC-TEL 2017, Tallinn), vol. 12, Sep 2017, pp. 625–628.

47

[11] C. Boura, P. Veiga, A. M. Santos, and A. Sardinha, “Napp - desenvolvimento de uma aplicacao

movel no ambito do programa de sucesso escolar,” in 4º Congresso Nacional de Praticas

Pedagogicas no Ensino Superior, CNaPPES 2017 (Setubal), accepted paper to be published, 2017.

[12] Chronus, Mentoring Software, Accessed November 22, 2016, http://chronus.com.

[13] Chronus, Mobile Applications, Accessed November 22, 2016, http://chronus.com/

news-and-events/chronus-introduces-industrys-comprehensive-mobile-experience.

[14] My Study Life, Accessed December 23, 2016, https://www.mystudylife.com.

[15] Focus Booster, Accessed December 23, 2016, https://www.focusboosterapp.com.

[16] M. Ruensuk, “An implementation to reduce internal/external interruptions in agile software develop-

ment using pomodoro technique,” in 2016 IEEE/ACIS 15th International Conference on Computer

and Information Science (ICIS), June 2016, pp. 1–4.

[17] C. P. Leao and A. C. Ferreira, “Shedding light on important mentorship relationships issues in

advanced engineering education,” in Engineering Education (CISPEE), 2013 1st International Con-

ference of the Portuguese Society for, Oct 2013, pp. 1–6.

[18] Nucleo de Apoio ao Estudante - NAPE, Accessed November 3, 2016, https://nape.tecnico.ulisboa.

pt/nape/.

[19] TU Delft - TPM, EPA and MOT Mentoring System, Accessed November 10, 2016, http://studenten.

tudelft.nl/en/students/faculty-specific/tpm/academic-counsellors/mentor-system/.

[20] TU Delft - AE Mentoring System, Accessed November 10, 2016, http://studenten.tudelft.nl/index.

php?id=104227.

[21] Carnegie Mellon University - Heinz College Mentor Programme, Accessed November 12, 2016,

http://www.heinz.cmu.edu/alumni/be-a-mentor/index.aspx.

[22] MIT Sloan School of Management Mentoring Programme, Accessed November 14, 2016, http:

//mitsloan.mit.edu/alumni/get-involved/mentoring-program/.

[23] KTH Student Mentoring Programme, Accessed November 14, 2016, https://www.kth.se/en/student/

framtid/karriar/mentorprogram/kth-mentorprogram-1.64307.

[24] University Of Michigan Office Of New Student Programs, Accessed November 14, 2016, http://

onsp.umich.edu/mentorship.

[25] Kit Normas Basicas de Identidade, Accessed December 2, 2016, https://tecnico.ulisboa.pt/files/

2015/07/Kit-Normas-IST.pdf.

48

[26] Google’s Progressive Web Apps, Accessed December 28, 2016, https://developers.google.com/

web/progressive-web-apps/.

[27] D. Sin, E. Lawson, and K. Kannoorpatti, “Mobile web apps - the non-programmer’s alternative to

native applications,” in 2012 5th International Conference on Human System Interactions, June

2012, pp. 8–15.

[28] A. Charland and B. Leroux, “Mobile application development: Web vs. native,” Commun. ACM,

vol. 54, no. 5, pp. 49–53, May 2011. [Online]. Available: http://doi.acm.org/10.1145/1941487.

1941504

[29] I. Dalmasso, S. K. Datta, C. Bonnet, and N. Nikaein, “Survey, comparison and evaluation of cross

platform mobile application development tools,” in 2013 9th International Wireless Communications

and Mobile Computing Conference (IWCMC), July 2013, pp. 323–328.

[30] P. J. Sadalage and M. Fowler, NoSQL distilled: a brief guide to the emerging world of polyglot

persistence. Pearson Education, 2012.

[31] E. Redmond and J. R. Wilson, Seven databases in seven weeks: a guide to modern databases

and the NoSQL movement. Pragmatic Bookshelf, 2012.

[32] Apache CouchDB, Accessed December 10, 2016, http://couchdb.apache.org/.

[33] CouchDB Replication Protocol, Accessed December 15, 2016, http://docs.couchdb.org/en/master/

replication/protocol.html.

[34] M. S. Turan, E. B. Barker, W. E. Burr, and L. Chen, “Sp 800-132. recommendation for password-

based key derivation: Part 1: Storage applications,” Gaithersburg, MD, United States, Tech. Rep.,

2010.

[35] R. Barata, S. Silva, D. Martinho, L. Cruz, and L. G. Silva, “Open apis in information systems for

higher education,” in EUNIS Journal of Higher Education IT Issue 2015-2.

[36] F. Nayebi, J. M. Desharnais, and A. Abran, “The state of the art of mobile application usability evalu-

ation,” in 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE),

April 2012, pp. 1–4.

[37] J. Gao, X. Bai, W. T. Tsai, and T. Uehara, “Mobile application testing: A tutorial,” Computer, vol. 47,

no. 2, pp. 46–55, Feb 2014.

[38] How To Use The System Usability Scale (SUS) To Evalu-

ate The Usability, Accessed June 13, 2017, http://usabilitygeek.com/

how-to-use-the-system-usability-scale-sus-to-evaluate-the-usability-of-your-website/.

49

50