The Advantages of Monitoring Sakai Usage
By
Jose L. Silva-LugoCarl J. Perry
John R. Boekenoogene-Learning Support Services
Office of Academic Technology University of Florida
04/14/2023
2
Introduction
The University of Florida (UF)’s contract with Blackboard WebCT was going to end in October 2010.
In Spring of 2009, UF came up with four options: Sakai, Blackboard, Moodle, and Angel.
UF ran open meetings with each vender.
UF set up test sites for faculty, staff, and students.
UF selected Sakai
04/14/2023
3
Introduction: Planning, Testing, & Implementation
2009 2010 2011 201
2
Process Dec Spr Sum Fall Spr Sum Fall Spr
Planning
Pilot Study
Migration
Introd. to UF
Roll Out
Workshops
Open Labs
Webinars
04/14/2023
4
Introduction: Training & Assistant Offer to UF
• Getting Started• Moving On• Evaluating knowledge with online assessment
• Begin a new semester• End a semester
Workshops
• 1 h consultation per week• There is no need of appointment• Every Tuesdays through the semester
Open labs
• 30 minutes per week• Every Wednesday through the semesterWebinar
04/14/2023
5
Introduction: The Need of Monitoring Sakai Usage
Even though the effort of promoting Sakai carried out from summer 2010 to 2011, we did not have the expected attendance.
We decided to look our records to: Find out the percentage of courses, sections, and
instructors using Sakai per college/year Identify the colleges with percentages less than 50% used
of Sakai Plot the percentage of courses, sections, and instructors
per semester Target colleges with a marketing program
04/14/2023
6
Methodology: Databases
Data was retrieved from the network: Registrar Office: IBM DB2 database Middleware: MySQL database
Language used to query the database: A PHP script queried both database (< 5
seconds). The same script merged and compared the
results (5-20 seconds).
04/14/2023
7
Methods: Points of Interest about Querying
You need to understand the database.
You have to find how to best compare the results: The Registrar DB2 had zero to many instructors associated with a
section The Registrar DB2 used postfix characters in the course name, such
as for labs
The results produced from each database were accurate and error free.
Manipulating large data sets and verifying the results were accurate.
04/14/2023
8
Methodology: Calculating Percentages
Registrar (Total) • Courses• Sections• Instructors
Middleware (Sakai)• Courses• Sections• Instructors
% of Courses Xx 100
04/14/2023
9
Methodology: Statistics
Statistical Analysis: Proportion comparison:
Between 2010 and 2011 for courses, sections, and instructors per college
Between semesters for courses, sections, and instructors
Alpha level = 0.05 Two-tailed test
04/14/2023
10
No. of Courses from the Registrar Office and Middleware in 2010 & 2011
2010 2011College Registrar Middleware Percentage Registrar Middleware PercentageDESIGN, CONSTRUCTION AND PLANNING 307 91 29.64 348 256 73.56HEALTH AND HUMAN PERFORMANCE 383 91 23.76 423 282 66.67ENGINEERING 1,100 218 19.82 1,127 445 39.49JOURNALISM AND COMMUNICATIONS 311 57 18.33 289 191 66.09MEDICINE 534 92 17.23 519 116 22.35PHARMACY 244 42 17.21 296 150 50.68LIBERAL ARTS AND SCIENCES 3,138 455 14.50 2,872 1,237 43.07PUBLIC HEALTH & HEALTH PROFESSIONS 457 65 14.22 458 245 53.49BUSINESS ADMINISTRATION 645 88 13.64 653 406 62.17AGRICULTURAL AND LIFE SCIENCES 1,602 213 13.30 1,647 573 34.79CROSS-COLLEGE 36 4 11.11 47 0 0.00FINE ARTS 1,022 56 5.48 1,037 179 17.26VETERINARY MEDICINE 259 13 5.02 264 51 19.32EDUCATION 630 18 2.86 662 54 8.16DENTISTRY 244 1 0.41 279 5 1.79ACCOUNTING 64 0 0.00 65 0 0.00BUILDING CONSTRUCTION 211 0 0.00 218 0 0.00CONTINUING EDUCATION 209 0 0.00 171 0 0.00LEVIN COLLEGE OF LAW 190 0 0.00 550 11 2.00MILITARY 62 0 0.00 55 0 0.00NURSING 133 0 0.00 130 88 67.69
Total 11,781 1,504 12.77 12,110 4,289 35.42
04/14/2023
11
04/14/2023
12
Courses: Proportion Comparison between 2010 and 2011 per College
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value ResultAGRICULTURAL 0.13 0.35 0.2419 0.0150 14.3034 1.645 < 0.0001 P < 0.05BUSINESS ADMINISTRATION 0.14 0.62 0.3806 0.0270 18.0054 1.645 < 0.0001 P < 0.05DENTISTRY 0.00 0.02 0.0115 0.0093 1.4809 1.645 0.0694 P > 0.05DESIGN, CONSTRUCTION 0.30 0.74 0.5298 0.0391 11.2387 1.645 < 0.0001 P < 0.05EDUCATION 0.03 0.08 0.0557 0.0128 4.1510 1.645 < 0.0001 P < 0.05ENGINEERING 0.20 0.39 0.2977 0.0194 10.1481 1.645 < 0.0001 P < 0.05FINE ARTS 0.05 0.17 0.1141 0.0140 8.4064 1.645 < 0.0001 P < 0.05HEALTH & HUMAN PERFORMANCE 0.24 0.67 0.4628 0.0352 12.2002 1.645 < 0.0001 P < 0.05JOURNALISM & COMMUNICATIONS 0.18 0.66 0.4133 0.0402 11.8711 1.645 < 0.0001 P < 0.05LEVIN COLLEGE OF LAW 0.00 0.02 0.0149 0.0102 1.9640 1.645 0.0250 P < 0.05LIBERAL ARTS & SCIENCES 0.14 0.43 0.2787 0.0115 25.0227 1.645 < 0.0001 P < 0.05MEDICINE 0.17 0.22 0.1975 0.0245 2.0872 1.645 0.0188 P < 0.05NURSING 0.00 0.68 0.3346 0.0582 11.6320 1.645 < 0.0001 P < 0.05PHARMACY 0.17 0.51 0.3556 0.0414 8.0846 1.645 < 0.0001 P < 0.05PUBLIC HEALTH 0.14 0.53 0.3388 0.0313 12.5489 1.645 < 0.0001 P < 0.05VETERINARY MEDICINE 0.05 0.19 0.1224 0.0287 4.9889 1.645 < 0.0001 P < 0.052010 0.13 0.50 0.3142 0.0060 61.6931 1.645 < 0.0001 P < 0.052011 0.35 0.50 0.4248 0.0064 22.2413 1.645 < 0.0001 P < 0.05
04/14/2023
13
04/14/2023
14
04/14/2023
15
Courses: Proportion Comparison between Semesters
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
Spring 2010 - Summer 2010 0.00 0.04 0.0166 0.0031 13.4318 1.645 < 0.0001 P < 0.05
Summer 2010 - Summer 2011 0.04 0.29 0.1772 0.0102 24.5078 1.645 < 0.0001 P < 0.05
Fall 2010 - Spring 2011 0.30 0.37 0.3344 0.0099 6.4253 1.645 < 0.0001 P < 0.05
Fall 2010 - Summer 2011 0.30 0.29 0.2985 0.0107 1.0054 1.645 0.1562 P > 0.05
Fall 2010 - Fall 2011 0.30 0.38 0.3432 0.0099 8.1293 1.645 < 0.0001 P < 0.05
Spring 2011 - Summer 2011 0.37 0.29 0.3363 0.0111 6.7244 1.645 < 0.0001 P < 0.05
Spring 2011 - Fall 2011 0.37 0.38 0.3752 0.0102 1.6609 1.645 0.0485 P < 0.05
Summer 2011 - Fall 2011 0.29 0.38 0.3469 0.0111 8.2315 1.645 < 0.0001 P < 0.05
04/14/2023
16Not significantly different
04/14/2023
17
No. of Sections from the Registrar Office and Middleware in 2010 & 2011
2010 2011College Registrar Middleware Percentage Registrar Middleware Percentage
HEALTH AND HUMAN PERFORMANCE 628 169 26.91 894 563 62.98MEDICINE 604 134 22.19 682 165 24.19DESIGN, CONSTRUCTION AND PLANNING 577 125 21.66 582 341 58.59ENGINEERING 2467 500 20.27 2384 784 32.89PHARMACY 718 131 18.25 735 426 57.96LIBERAL ARTS AND SCIENCES 7653 1308 17.09 7632 3649 47.81CROSS-COLLEGE 66 11 16.67 71 0 0.00JOURNALISM AND COMMUNICATIONS 588 98 16.67 638 327 51.25BUSINESS ADMINISTRATION 1264 185 14.64 1306 915 70.06AGRICULTURAL AND LIFE SCIENCES 3220 457 14.19 3870 1144 29.56PUBLIC HEALTH & HEALTH PROFESSIONS 811 90 11.10 919 347 37.76FINE ARTS 1843 87 4.72 1975 273 13.82VETERINARY MEDICINE 325 14 4.31 370 63 17.03EDUCATION 1299 22 1.69 1389 78 5.62DENTISTRY 356 1 0.28 372 4 1.08ACCOUNTING 136 0 0.00 116 0 0.00BUILDING CONSTRUCTION 269 0 0.00 277 0 0.00CONTINUING EDUCATION 228 0 0.00 177 0 0.00LEVIN COLLEGE OF LAW 441 0 0.00 503 11 2.19MILITARY 90 0 0.00 79 0 0.00NURSING 279 0 0.00 279 208 74.55
Total 23,862 3,332 13.96 25,250 9,298 36.82
04/14/2023
18
04/14/2023
19
Sections: Proportion Comparison between 2010 and 2011 per College
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value ResultAGRICULTURAL 0.14 0.30 0.2258 0.0100 15.4094 1.645 < 0.0001 P < 0.05BUSINESS ADMINISTRATION 0.15 0.70 0.4280 0.0195 28.3899 1.645 < 0.0001 P < 0.05DENTISTRY 0.00 0.01 0.0069 0.0061 1.2973 1.645 0.0968 P > 0.05DESIGN, CONSTRUCTION & PLANNING 0.22 0.59 0.4021 0.0288 12.8197 1.645 < 0.0001 P < 0.05EDUCATION 0.02 0.06 0.0372 0.0073 5.3689 1.645 < 0.0001 P < 0.05ENGINEERING 0.20 0.33 0.2647 0.0127 9.9592 1.645 < 0.0001 P < 0.05FINE ARTS 0.05 0.14 0.0943 0.0095 9.6172 1.645 < 0.0001 P < 0.05HEALTH & HUMAN PERFORMANCE 0.27 0.63 0.4809 0.0260 13.8633 1.645 < 0.0001 P < 0.05JOURNALISM & COMMUNICATIONS 0.17 0.51 0.3467 0.0272 12.7130 1.645 < 0.0001 P < 0.05LEVIN COLLEGE OF LAW 0.00 0.02 0.0117 0.0070 3.1238 1.645 < 0.001 P < 0.05LIBERAL ARTS & SCIENCES 0.17 0.48 0.3243 0.0076 40.5675 1.645 < 0.0001 P < 0.05MEDICINE 0.22 0.24 0.2325 0.0236 0.8508 1.645 0.1917 P > 0.05NURSING 0.00 0.75 0.3728 0.0409 18.2102 1.645 < 0.0001 P < 0.05PHARMACY 0.18 0.58 0.3833 0.0255 15.5669 1.645 < 0.0001 P < 0.05PUBLIC HEALTH & HEALH PROFESSIONS 0.11 0.38 0.2526 0.0209 12.7359 1.645 < 0.0001 P < 0.05VETERINARY MEDICINE 0.04 0.17 0.1108 0.0239 5.3304 1.645 < 0.0001 P < 0.052010 0.14 0.50 0.3213 0.0043 84.9707 1.645 < 0.0001 P < 0.052011 0.37 0.50 0.4316 0.0044 28.7675 1.645 < 0.0001 P < 0.05
04/14/2023
20
04/14/2023
21
04/14/2023
22
Sections: Proportion Comparison between Semesters
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
Spring 2010 - Summer 2010 0.00 0.05 0.0163 0.0023 21.5659 1.645 < 0.0001 P < 0.05
Summer 2010 - Summer 2011 0.05 0.24 0.1576 0.0074 25.3576 1.645 < 0.0001 P < 0.05
Fall 2010 - Spring 2011 0.31 0.40 0.3528 0.0068 12.3286 1.645 < 0.0001 P < 0.05
Fall 2010 - Summer 2011 0.31 0.24 0.2850 0.0075 9.7743 1.645 < 0.0001 P < 0.05
Fall 2010 - Fall 2011 0.31 0.42 0.3636 0.0068 15.2238 1.645 < 0.0001 P < 0.05
Spring 2011 - Summer 2011 0.40 0.24 0.3371 0.0079 19.8939 1.645 < 0.0001 P < 0.05
Spring 2011 - Fall 2011 0.40 0.42 0.4058 0.0070 2.7055 1.645 0.0034 P < 0.05
Summer 2011 - Fall 2011 0.24 0.42 0.3514 0.0079 22.3055 1.645 < 0.0001 P < 0.05
04/14/2023
23
No. of Instructors from the Registrar Office and Middleware in 2010 & 2011
2010 2011College Registrar Middleware Percentage Registrar Middleware Percentage
CROSS-COLLEGE 36 10 27.78 37 0 0.00HEALTH AND HUMAN PERFORMANCE 220 56 25.45 267 175 65.54ENGINEERING 697 176 25.25 758 417 55.01DESIGN, CONSTRUCTION AND PLANNING 218 46 21.10 236 129 54.66BUSINESS ADMINISTRATION 405 80 19.75 414 124 29.95JOURNALISM AND COMMUNICATIONS 286 56 19.58 318 180 56.60LIBERAL ARTS AND SCIENCES 3,364 519 15.43 3,843 1,289 33.54PUBLIC HEALTH & HEALTH PROFESSIONS 403 61 15.14 444 162 36.49AGRICULTURAL AND LIFE SCIENCES 1,003 147 14.66 1,050 388 36.95FINE ARTS 621 50 8.05 659 141 21.40VETERINARY MEDICINE 256 14 5.47 489 32 6.54MEDICINE 324 16 4.94 541 49 9.06EDUCATION 583 21 3.60 581 67 11.53PHARMACY 164 3 1.83 236 8 3.39DENTISTRY 132 1 0.76 571 4 0.70ACCOUNTING 40 0 0.00 45 0 0.00BUILDING CONSTRUCTION 67 0 0.00 82 0 0.00CONTINUING EDUCATION 12 0 0.00 5 0 0.00LEVIN COLLEGE OF LAW 256 0 0.00 297 9 3.03MILITARY 28 0 0.00 31 0 0.00NURSING 174 0 0.00 205 6 2.93
Total 9,289 1,256 13.52 11,109 3,180 28.63
04/14/2023
24
04/14/2023
25
Instructors: Proportion Comparison between 2010 and 2011 per College
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value ResultAGRICULTURAL 0.15 0.37 0.2606 0.0194 11.5043 1.645 < 0.0001 P < 0.05BUSINESS ADMINISTRATION 0.20 0.30 0.2491 0.0302 3.3741 1.645 < 0.001 P < 0.05DENTISTRY 0.01 0.01 0.0071 0.0081 0.0703 1.645 0.4721 P > 0.05DESIGN, CONSTRUCTION 0.21 0.55 0.3855 0.0457 7.3403 1.645 < 0.0001 P < 0.05EDUCATION 0.04 0.12 0.0756 0.0155 5.1170 1.645 < 0.0001 P < 0.05ENGINEERING 0.25 0.55 0.4076 0.0258 11.5416 1.645 < 0.0001 P < 0.05FINE ARTS 0.08 0.21 0.1492 0.0199 6.6968 1.645 < 0.0001 P < 0.05HEALTH & HUMAN PERFORMANCE 0.25 0.66 0.4743 0.0455 8.8171 1.645 < 0.0001 P < 0.05JOURNALISM & COMMUNICATIONS 0.20 0.57 0.3907 0.0398 9.3113 1.645 < 0.0001 P < 0.05LEVIN COLLEGE OF LAW 0.00 0.03 0.0163 0.0108 2.8082 1.645 0.0025 P < 0.05LIBERAL ARTS & SCIENCES 0.15 0.34 0.2509 0.0102 17.6964 1.645 < 0.0001 P < 0.05MEDICINE 0.05 0.09 0.0751 0.0185 2.2242 1.645 0.0132 P < 0.05NURSING 0.00 0.03 0.0158 0.0129 2.2748 1.645 0.0116 P < 0.05PHARMACY 0.02 0.03 0.0275 0.0166 0.9387 1.645 0.1736 P > 0.05PUBLIC HEALTH 0.15 0.36 0.2633 0.0303 7.0459 1.645 < 0.0001 P < 0.05VETERINARY MEDICINE 0.05 0.07 0.0617 0.0186 0.5791 1.645 0.2810 P > 0.052010 0.14 0.50 0.3152 0.0068 52.8065 1.645 < 0.0001 P < 0.052011 0.50 0.29 0.3907 0.0065 31.9009 1.645 < 0.0001 P < 0.05
04/14/2023
26
04/14/2023
27
04/14/2023
28
Instructors: Proportion Comparison between Semesters
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
Spring 2010 - Summer 2010 0.00 0.07 0.0235 0.0044 14.5203 1.645 < 0.0001 P < 0.05
Summer 2010 - Summer 2011 0.07 0.23 0.1582 0.0113 14.3211 1.645 < 0.0001 P < 0.05
Fall 2010 - Spring 2011 0.29 0.32 0.3065 0.0103 3.5117 1.645 < 0.001 P < 0.05
Fall 2010 - Summer 2011 0.29 0.23 0.2645 0.0114 5.3643 1.645 < 0.0001 P < 0.05
Fall 2010 - Fall 2011 0.29 0.28 0.2856 0.0099 0.4479 1.645 0.3264 P > 0.05
Spring 2011 - Summer 2011 0.32 0.23 0.2881 0.0115 8.4099 1.645 < 0.0001 P < 0.05
Spring 2011 - Fall 2011 0.32 0.28 0.3030 0.0099 4.0899 1.645 < 0.0001 P < 0.05
Summer 2011 - Fall 2011 0.23 0.28 0.2636 0.0111 5.1051 1.645 < 0.0001 P < 0.05
04/14/2023
29 Not significantly different
Significantly different
04/14/2023
30
Discussion
In general, course, sections, and instructors using Sakai are not monitored.
This simple procedure has the following advantages: Provide information about the proportion of
instructors, courses, and sections using Sakai. It helps to monitor the implementation process. It creates the database for marketing programs. It might be used to evaluate success at the college
and at university level.
04/14/2023
31
Conclusions
There was significant increase in their proportions of courses, sections, and instructors from 2010 to 2011 in the
majority of the colleges.
There was significant increase in their proportions of courses, sections, and
instructors per semester.
We have identified the colleges that need more our attention, and we are targeting
them with a marketing program.
Monitoring Sakai usage is important during the implementation and evaluation of the
overall success at the college and university level.
Top Related