8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan...
-
Upload
russell-packwood -
Category
Documents
-
view
218 -
download
0
Transcript of 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan...
![Page 1: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/1.jpg)
8th Sakai Conference
4-7 December 2007Newport Beach
What does Quality Assurance What does Quality Assurance Mean to Sakai?Mean to Sakai?
Alan Berg
Megan May
Seth Theriault
![Page 2: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/2.jpg)
TopicsTopics• What does a QA Director do?• QA Today
– Who is in the WG– Types of testing currently done and release activities– Strides forward
• Resources available – QA Network
• 2.5 Release Activities Update• QA Tomorrow• Questions
![Page 3: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/3.jpg)
Role of QA DirectorRole of QA Director
• Coordination! – Help bring parties interested in the same testing
efforts together– Foster initiatives that support quality throughout
the community– Assist in bridging the gap between test and
design & development teams– Ensure release activities are transparent
![Page 4: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/4.jpg)
QA Today QA Today
Development
Testing & Software Release
Requirement Gathering and
Design
Focus on Functional Testing
![Page 5: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/5.jpg)
Who is the QA WG?Who is the QA WG?
2.4.0
93 People
37 Institutions
11 Countries
2.2.1
23 People
10 Institutions
4 Countries
2.3.0
44 People
17 Institutions
5 Countries
2.5.0
56 People
17 Institutions
4 Countries
. . . How do they get involved?. . . How do they get involved?
![Page 6: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/6.jpg)
Strides ForwardStrides Forward
• Deployment testing on wider variety of platforms
• Integration of Accessibility WG activities in general QA cycle
• Expanding/updating library of test scripts• Reflection on Release process
– http://confluence.sakaiproject.org/confluence/x/fbU
![Page 7: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/7.jpg)
QA NetworkQA Network
Amsterdam Boston U
Cambridge Cape Town
Columbia Indiana
rSmart Georgia Tech
Coming soon servers in Japan and Australia
![Page 8: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/8.jpg)
CollabCollabHow to be in the know!How to be in the know!https://collab.sakaiproject.org/portal
![Page 10: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/10.jpg)
ConfluenceConfluencehttp://confluence.sakaiproject.org/confluence/display/QA/Home
![Page 11: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/11.jpg)
Perceived Quality LevelPerceived Quality LevelDevelopment
Management
QA
Support
Overall Perceived level of Quality
![Page 12: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/12.jpg)
Quality, like beauty, is clearly in the eyes of the beholder.
![Page 13: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/13.jpg)
Perceived Quality PollPerceived Quality Poll
User Perceptions
![Page 14: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/14.jpg)
QA Tomorrow QA Tomorrow
• Sakai Community Concerns – Several prominent campuses experience serious
performance problems with 2.4
• 2.5 Release Update– Further refinement of process– Goals for 2.5
![Page 15: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/15.jpg)
Paths ForwardPaths Forward
• Common set of testing vocabulary – Unit Testing– Integration testing– System Testing– Performance
• Load testing ([email protected]) • Integration testing
Development
Testing & Software Release
Requirement Gathering and Design
http://confluence.sakaiproject.org/confluence/x/M4B5
![Page 16: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/16.jpg)
Roadmap for TomorrowRoadmap for Tomorrow
![Page 17: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/17.jpg)
Metrics to Describe QualityMetrics to Describe Quality
2.3.0 – 85% Verification
2.2.0 – 85% Verification
(1,345 JIRA Tickets)
(611 JIRA Tickets)
(504 JIRA Tickets)
2.4.0 – 82% Verification
2.5.0 – Past bug verification
![Page 18: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/18.jpg)
Metrics to Describe QualityMetrics to Describe Quality
Unit Integration System PerformanceKernelAssignmentsGradebookResources
![Page 19: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/19.jpg)
Static code ReviewStatic code Review• Search for bug patterns• What to process: Source code, Binary• Two types of bug pattern
– Generic• Classic (buffer overflow, input validation)
– Contextual• Language specific• Framework specific• Configuration specific
• Proximity effect
![Page 20: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/20.jpg)
Hinting nightlyHinting nightly
http://qa1-nl.sakaiproject.org
![Page 21: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/21.jpg)
Random ExampleRandom Example
![Page 22: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/22.jpg)
Duplicate CodeDuplicate Code
![Page 23: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/23.jpg)
Do we wish to use the reports more Do we wish to use the reports more aggressively?aggressively?
• Are the developers taking note?– Bugs reported are not always correct– Bugs found not always important– But duplicate code, unit test coverage, and failing to deal properly
with exceptions hint strongly at quality.– Not enough Contextual searching for Sakai and spring
configuration specific issues– Reports do not explain enough– Answer probably not, but thankfully trends in the marketplace will improve
the quality of results overtime.
![Page 24: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/24.jpg)
A year onA year on
• Update prototype– Less noise more explanation– Graphical tracking of change
• Recommend Eclipse Plugins
• Look at emerging products
• See as part of a series of QA processes.
• Agree on metrics to act on and then act.
![Page 25: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/25.jpg)
A good book to readA good book to read
• Secure Programming with Static Analysis– Brian Chess, Jacob West
• Addison Wesley
• ISBN: 9780321424778
• Includes CD with software and examples.
![Page 26: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/26.jpg)
What you can doWhat you can do• Pledge Resources
– What happens in the community has a direct local impact– Easy point of entry into Sakai– All skill sets are needed
• Discussion– Why hasn’t this been a priority?– What are obstacles in the way and how do we remove
them?
![Page 27: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/27.jpg)
![Page 28: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/28.jpg)
Questions
![Page 29: 8th Sakai Conference4-7 December 2007 Newport Beach What does Quality Assurance Mean to Sakai? Alan Berg Megan May Seth Theriault.](https://reader035.fdocuments.net/reader035/viewer/2022062318/551b8c65550346942b8b4de7/html5/thumbnails/29.jpg)
Other Sessions of InterestOther Sessions of Interest
• 2.5 QA Workshop (Thursday, 13:00 - 14:30 in Salon 5)
• The Automated Testing Conundrum in Sakai (Thursday, 15:10 - 16:10 in Salon 5)