DOMjudge · 2020. 11. 23. · the ACM ICPC fixed problem set and timeframe has been used in several...
Transcript of DOMjudge · 2020. 11. 23. · the ACM ICPC fixed problem set and timeframe has been used in several...
Programming Systems Group • Martensstr. 3 • 91058 Erlangen • Germany
DOMjudgeAn automated judge system for programming contests
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
DOMjudge in a nutshell
follows KISS principle depends on standard software (MySQL, Apache, PHP) uses a simple web interface as frontend for teams, jury
and general public□problem submission□clarification requests□(re)judging□...
provides a modular system for plugging in different programming languages
supports distributed and fully automated judging stores problems, submissions, testdata in a database
2
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
DOMjudge - where to use it?
designed to be used in programming contests likethe ACM ICPC
fixed problem set and timeframe has been used in several live contests□North Western European Regional 2007-2009□South Western European Regional 2008□Benelux Algorithm Programming Contest 2004,’06,’07,’09□several local contests
not designed for online submission systems,but we (and others) have a modified version
3
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
DOMjudge is Open Source Software
originally developed by the Study Association of the University Utrecht
licensed under the GNU General Public License modified versions:□P.J.S. University in Kosice (courseware)□ETH Zurich (automatic exam judging software)□FAU Erlangen-Nuremberg (courseware)
DOMjudge is under active development Patches with bug fixes / new features from forks are
applied to the original tree no security through obscurity, but true security as
important design goal
4
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Requirements
5
web/db-server
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Requirements
5
web/db-server
team and jury workstations
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Requirements
5
web/db-server
one or more judgehosts
team and jury workstations
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Scoreboard
6
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Scoreboard
6
# solved problems /
time
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Scoreboard
6
# solved problems /
time
#submissions (time + penalty
time)
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Scoreboard
6
# solved problems /
time
#submissions (time + penalty
time)
possibility to freeze/ unfreeze
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Scoreboard
6
# solved problems /
time
#submissions (time + penalty
time)
possibility to freeze/ unfreeze
all data on scoreboard is cached
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Scoreboard
6
# solved problems /
time
#submissions (time + penalty
time)
possibility to freeze/ unfreeze
all data on scoreboard is cached
cells are clickable & filter
submissions
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Programming Languages
7
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Programming Languages
7
Autodetect language by suffix
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Programming Languages
7
Autodetect language by suffix
factor for slower languages
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Problems Overview
8
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Problems Overview
8
old problems
grayed out
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Problems Overview
8
old problems
grayed out
discontinuesubmissions/
judgings
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Edit/Create Problems
9
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Edit/Create Problems
9
corrector program that follows the ICPC
validator standard
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Edit/Create Problems
9
corrector program that follows the ICPC
validator standard
multiple testcases stored in database
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Edit/Create Problems
9
corrector program that follows the ICPC
validator standard
multiple testcases stored in database
upload problem incl. test data aszip archive (wip)
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Problem Overview
10
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Problem Overview
10
rejudge problems(e.g. in case of
invalid test data)
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Testcase Overview
11
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Testcase Overview
11
change order in which the testcases
are judged
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Team Overview
12
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Team Overview
12
sort tables with javascript
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Team Overview
12
sort tables with javascript
strict IP checking possible
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Team Overview
12
sort tables with javascript
strict IP checking possible
status light for login, submissions, AC
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Clarification Requests
13
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Clarification Requests
13
new requests are highlighted
with AJAX
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Clarification Requests
14
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Balloon Interface
15
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission Overview
16
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission Overview
16
optional manual verification
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission Overview
16
optional manual verificationignored submission
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission Overview
16
optional manual verificationignored submission
status of judge queue
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
FAU: Submission Overview Changes
17
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
FAU: Submission Overview Changes
17
filter options
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
FAU: Submission Overview Changes
17
filter options
judge tracking
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission
18
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission
18
multiple judgings
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission
18
multiple judgings
stored in cookie
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Single testcase result
19
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Single testcase result
19
line-by-line diff
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Single testcase result
19
line-by-line diff
FAU: LCS diff
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
View source code
20
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
View source code
20
syntax highlighting
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
View source code
20
syntax highlighting
diff to last submission
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
View source code
20
syntax highlighting
diff to last submission
download, or edit & resubmit source
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submit via Webinterface
21
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submit via Webinterface
21
mini-scoreboard on every page
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submit via Webinterface
21
select language by filename suffix
mini-scoreboard on every page
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submit via Webinterface
21
select language by filename suffix
alternatively submit via
command line
mini-scoreboard on every page
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission details
22
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Submission details
22
can be configured
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Judging process
start judgedaemon judgedaemon connects to database for every submission:□poll source code□setup chroot□compile source, (+ create run script if necessary)□for every testcase:
poll test datarun submission with test data (with runguard, limited resources and special user)
compare results (plain diff or special corrector)write verdict of single run back to database
□combine verdicts, write back to database23
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Testdata
multiple testcases per problem stored in database testcase cache on each judgehost based on md5sum verdicts have an assigned priority□verdicts with higher priority are used in combination□equal priority: use first verdict□optional: returns final result as soon as one verdict with
highest priority is found (e.g. TLE, MLE, RTE...) Future Plans: remap results (e.g. PE to AC or PE to
WA)
24
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Plugin
output scoreboard, events in XML format can be used e.g. for external scoreboards
25
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
DOMjudge@FAU
26
DOMjudge is used as courseware, for contests and training of our ICPC teams
“Hello World - for advanced students”:□weekly algorithm presentations during summer term□five or more corresponding problems of varying difficulty□solved in supervised training sessions or at home□credit points for presentation and solved problems
Training:□training contests at regular intervals□parallel training set in DOMjudge□problems can be easily moved from onsite contest to
training set
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Do try this at home!
27
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Do try this at home!
check out http://domjudge.sourceforge.net for□source code download□further documentation□live demonstration of web frontend
27
DOMjudge - An automated judging system • T. Werth • University of Erlangen-Nuremberg, Germany • Slide
Do try this at home!
check out http://domjudge.sourceforge.net for□source code download□further documentation□live demonstration of web frontend
Thank you for your attention! Questions?
27