QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ......
Transcript of QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ......
![Page 1: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/1.jpg)
QA: Under the hood
ROME | 11 October 2017
Xisco Fauli - x1sc0
![Page 2: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/2.jpg)
2
Agenda● Automated Testing
● Statistics Page
● Bugzilla Tweaks
![Page 3: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/3.jpg)
Automated testing
![Page 4: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/4.jpg)
4
Overview
● Based on ‘office-interoperability-tools’ created by Milos Ramek
● Testing master builds since February 2017
● Run once a month on vm174
● Finding regressions and progressions (new) in documents
● Used for Writer and Impress
● Link: https://github.com/x1sc0/office-interoperability-tools
![Page 5: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/5.jpg)
5
Writer
EXPORT
ROUNDTRIP
IMPORT
IMPORT
EXPORT
PDF COMPARISON
ORIGINAL
![Page 6: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/6.jpg)
6
IMPORT
Impress
EXPORTROUNDTRIP
IMPORT
PDF COMPARISON
ORIGINAL
![Page 7: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/7.jpg)
7
Original files
● 757 PPTX files
● 1642 PPT files
● 610 RTF files
● 2525 DOC files
● 2304 DOCX files
![Page 8: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/8.jpg)
8
Example 1
![Page 9: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/9.jpg)
9
Example 2
![Page 10: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/10.jpg)
10
Example 3
![Page 11: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/11.jpg)
11
Exemple 4
![Page 12: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/12.jpg)
12
Results
● ~120 regressions found
● 8 progressions found (new)
– Moved to RESOLVED WORKSFORME
![Page 13: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/13.jpg)
13
Future ideas
● Add support for PPT/PPTX → ODP
● Test ODF documents
– Compare Import and export files?
![Page 14: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/14.jpg)
Statistics Page
![Page 15: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/15.jpg)
15
Overview
● Tries to acknowledge and give visibility to the effort people put in QA
● Tries to collect the data we have about QA in one place
● Link: https://wiki.documentfoundation.org/QA/Bugzilla/Statistics
![Page 16: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/16.jpg)
16
Time
![Page 17: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/17.jpg)
17
Version
![Page 18: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/18.jpg)
18
ESC meetings
● QA stats from ESC meetings from January 2016 up to today
● Good for checking how numbers evolve over time
– Unconfirmed bugs
– Open regressions
– Total number of regressions
– Open ‘bibisected’ bugs
– Total number of ‘bibisected’ bugs
– Open ‘bisected’ bugs
– Total number of ‘bisected’ bugs
– Reported bugs
![Page 19: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/19.jpg)
19
Weekly reports
● QA reports listed here since October, 2016
● Provides detailed information about who does what
● Priority changed
● System changed
● Bugs added to metabugs
● Bugs removed from metabugs
● ‘backportRequest’ added
● ‘backportRequest’ removed
● Bugs reported
● Nº of reported bugs still unconfirmed
● Total unconfirmed bugs
● Statuses changed
● Keywords added
● Keywords removed
● Severity changed
![Page 20: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/20.jpg)
Bugzilla tweaks
![Page 21: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/21.jpg)
21
Overview
● ‘QA-tools.py’ script
● Takes information from bugzilla_dump.json in vm144
● Find inconsistencies in Bugzilla
● Automate tasks
● Link: https://cgit.freedesktop.org/libreoffice/contrib/dev-tools/log/esc-reporting/qa-tools.py
![Page 22: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/22.jpg)
22
Auto-confirmed
PROBLEM
The reporter has auto-confirmed his/her own bug.
SOLUTION
Move the bug back to UNCONFIRMED
![Page 23: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/23.jpg)
23
Version
PROBLEM
Someone has changed the version field to a newer one.
SOLUTION
Move the bug back to the previous version
![Page 24: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/24.jpg)
24
Crash signature
PROBLEM
In order to link Bugzilla with http://crashreport.libreoffice.org, the crashreport field should have the syntax ["-signature-"], otherwise it doesn't work.
SOLUTION
Change it accordingly.
![Page 25: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/25.jpg)
25
RESOLVED FIXED
PROBLEM
The reporter has moved the bug to RESOLVED FIXED.
SOLUTION
The bug should be moved to RESOLVED WORKSFORME unless there's a commit fixing it.
![Page 26: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/26.jpg)
26
ASSIGNED
PROBLEM
Someone has taken the bug by changing the Assignee field but the status is not ASSIGNED.
SOLUTION
Change the status to ASSIGNED
![Page 27: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/27.jpg)
27
ASSIGNED
PROBLEM
Assignee field has been changed to default but the status is still ASSIGNED.
SOLUTION
Change the status back to NEW
![Page 28: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/28.jpg)
28
Assignee
PROBLEM
Status has changed to ASSIGNED but the assignee field hasn’t changed
SOLUTION
Change the Assignee field to the person who changed the status
![Page 29: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/29.jpg)
29
Assignee
PROBLEM
Status has changed back to NEW but the assignee field hasn’t changed
SOLUTION
Change the Assignee field to default
![Page 30: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/30.jpg)
30
Bug fixed ?
PROBLEM
Bug’s last comment is more than 30 days old && it’s from the ‘Commit Notification’
SOLUTION
Ask the developer to close the bug if it’s fixed
![Page 31: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/31.jpg)
31
Inactive assignment
PROBLEM
The bug has been in ASSIGNED status for more than 3 months without any activity.
SOLUTION
Change the status back to NEW and the assignee to default
![Page 32: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/32.jpg)
32
Inactive assignment
PROBLEM
The bug has been in ASSIGNED status for more than 3 months without any activity.
SOLUTION
Change the status back to NEW and the assignee to default
![Page 33: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/33.jpg)
33
New Contributors
PROBLEM
A new contributor has signed up in Bugzilla within the last month and he/she has interacted in more than 2 bugs.
SOLUTION
Send a welcoming email
![Page 34: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/34.jpg)
34
Patch Added
PROBLEM
A patch has been attached to Bugzilla along with the keyword 'patch'.
SOLUTION
Ask the user to submit the patch to Gerrit instead.
![Page 35: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/35.jpg)
35
Obsolete comments
PROBLEM
old bulk comments ( which are normally long ) are unpleasant.
SOLUTION
More than 10900 comments marked as obsolete so far.
![Page 36: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/36.jpg)
36
Future Ideas
● Make the bulk ‘untouched for a year’ comment totally automated
– Run it everyday instead of every month
– Hope more people will retest!
● Find contributors getting inactive after X time
● Ideas welcomed!
![Page 37: QA: Under the hood - LibreOffice · QA stats from ESC meetings from January 2016 up to today ... Change the status back to NEW and the assignee to default. 33 New Contributors PROBLEM](https://reader036.fdocuments.net/reader036/viewer/2022070916/5fb637ce2b05521e0c008ff4/html5/thumbnails/37.jpg)
All text and image content in this document is licensed under the Creative Commons Attribution-Share Alike 4.0 License (unless otherwise specified). "LibreOffice" and "The Document Foundation" are registered trademarks. Their respective logos and icons are subject to international copyright laws. The use of these therefore is subject to the trademark policy.
Thank you!