Response to Intervention Motivation in the Classroom: A Five-Part Framework Jim Wright .
Acctesting Framework - Motivation, Overview and First Experience
description
Transcript of Acctesting Framework - Motivation, Overview and First Experience
![Page 1: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/1.jpg)
1
Acctesting Framework - Motivation, Overview and First Experience
Kajetan FuchsbergerTE-MPE-TM, 2012-05-10
Thanks to:M. Galetzka, V.Baggiolini, R.Gorbonosov, M. Pojer,
M. Solfaroli Camillocci, M. Zerlauth
![Page 2: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/2.jpg)
K. Fuchsberger 2TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Content
1. Introduction/Motivation
2. System Overview
3. First Experience
![Page 3: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/3.jpg)
K. Fuchsberger 3TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Content
1. Introduction/Motivation
2. System Overview
3. First Experience
![Page 4: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/4.jpg)
K. Fuchsberger 4TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
LHC Hardware Comissioning
• About 7000 Tests on Magnet Circuits• Must be executed every year after Christmas
Stop (Potentially more after LS1)• Workflow:
1. ‘Execution’: Test Sequence is Executed on the Hardware Commissioning Sequencer.
2. ‘Analysis’: Test-Data (Measured Signals) is analysed (manually or partly automatic).
3. ‘Signing’: Experts have to ‘sign’ the test result (If manually analysed).
![Page 5: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/5.jpg)
K. Fuchsberger 5TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Motivation I
Initial Project Description: “Create replacement for the p2n Web-Page
aka Alvaro’s pages”
• PHP, grown over time Hard to maintain
![Page 6: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/6.jpg)
K. Fuchsberger 6TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Motivation II
• Some tests should be allowed to be executed even if previous analysis was not completed.
• Avoid starting of test if system is not ready.(e.g. locked)
• Avoid starting of tests on Circuits on same QPS controller.
• Force starting of tests on QF, QD at the same time
• ….. and some more …
New Requirements:
![Page 7: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/7.jpg)
K. Fuchsberger 7TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Motivation III
The HWC Legacy System:
• Many interdependent Systems:• Sequencer• DB (Central Point)• web page• Daemon• Fesa Class• LabView• …
Decision: Design a new system to orchestrate the whole process (and can replace several parts).
![Page 8: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/8.jpg)
K. Fuchsberger 8TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Content
1. Introduction/Motivation
2. System Overview
3. First Experience
![Page 9: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/9.jpg)
K. Fuchsberger 9TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Overview
e.g.: LabView, Powering Server …
e.g.: HWC Sequencer,
LHC Sequencer orCustom Components
e.g.: Magnet Circuits, BIC/PIC …
![Page 10: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/10.jpg)
K. Fuchsberger 10TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Acctesting Server
• Orchestrates the whole process:• Test Execution• Test Analysis
• Exclusively reads/persists data in the database.
• Notifies all the GUIs about changes.• Robust Design:
• Continuously persists relevant data to be able to recover in case of a crash.
• Gracefully handles unexpected behaviour of Execution- and Analysis Components.
![Page 11: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/11.jpg)
K. Fuchsberger 11TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Workflow I
1. User “expresses his wish” to execute one (or many) tests.
![Page 12: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/12.jpg)
K. Fuchsberger 12TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Workflow II
2. The Tests go to the Execution Basket (Server!)
3. The Scheduler (on the Server) will decide when to start which test(s).
![Page 13: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/13.jpg)
K. Fuchsberger 13TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Why so complicated?
• Central Scheduling can respect all the conditions, even if requests come from different GUIs.
• When conditions are fulfilled later, the tests are started automatically (No delays).
• No Need for reservation of circuits anymore. More dynamic
![Page 14: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/14.jpg)
K. Fuchsberger 14TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Test Phases
Tests within a test phase can be executed in arbitrary order.
TestPhase contains one or more Tests
![Page 15: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/15.jpg)
K. Fuchsberger 15TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Preconditions
• Before: Only email exchange, that e.g. “Cryo is ready for a certain group of circuits”.• Hard to track (History)• No real constraint on test execution ( Many
wrongly started tests in previous years)• Now: ‘SignOnlyTest’ for each precondition.
Automated Tests
![Page 16: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/16.jpg)
K. Fuchsberger 16TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Designed for Extension
• Server:• TestStepHandler (handle certain types of
SystemTests)• Constraints (Restrict Test Execution)• LockProvider (PIC, Db, …)• SystemInformationProvider (e.g. Issues)
• GUI:• TestResultsViewer (E.g. Powering Server)
Extension Points:
![Page 17: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/17.jpg)
K. Fuchsberger 17TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Test Step Handlers
• Responsible for executing a specific TestStep (execution, analysis) for a certain type of tests.
• Is itself responsible for communicating with other systems, if required.
• Examples:• HwcTestExecutionHandler: Communicates with HWC
Sequencer to execute the tests• DaemoneAnalysisHandler: Communicates with
LabView system to retrieve analysis results• … Future: PicBicTestExecutionHandler?
![Page 18: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/18.jpg)
K. Fuchsberger 18TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Constraints
• Simple Extension Point, which has to decide if one Test is allowed to be run together with another one. (Simple Yes/No decision)
• Used to formulate requirements like:“Only start one test on one of the four circuits on the same QPS controller.”
• Checked by the Scheduler, to decide if a certain test-configuration is allowed or not. See Michael’s presentation.
![Page 19: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/19.jpg)
K. Fuchsberger 19TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Soft Migration
• Working System had to be in place for start up 2012, but:• Old System should still work.• The switch between the two Systems should be easy.
• Achieved by:• Using the old Db-Schema for new System (some
restrictions!) + some (careful) extensions• WebPage stayed operational.
![Page 20: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/20.jpg)
K. Fuchsberger 20TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Content
1. Introduction/Motivation
2. System Overview
3. First Experience
![Page 21: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/21.jpg)
K. Fuchsberger 21TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Some Statistics
4 big Campaigns:
Min 5 Tests launched.
![Page 22: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/22.jpg)
K. Fuchsberger 22TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Test Statistics
Test failure rate did not decrease much !?
Average number of Tests per shift
Percentage of failed tests that timed out.
(Preconditions!)Efficiency increased.
![Page 23: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/23.jpg)
K. Fuchsberger 23TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Time between Tests?
• Expectation: Average time between test executions on one circuit should decrease!? Not evident !?
In the shadow of Analysis Time!
Tim
e [h
]
(Data: X-Mas 11/12)
![Page 24: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/24.jpg)
K. Fuchsberger 24TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Analysis Time
PIC2 and PNO.a1 automated (if successful)
![Page 25: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/25.jpg)
K. Fuchsberger 25TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
HWC Campaign 2012
• System was operational from the beginning. No big problems. Never stopped progress.
• GUI was very well accepted (Lot of positive feedback).• Preconditions were useful, at least for tracking. Users
Point of view?• Constraints turned out to be very useful although
sometimes puzzling. Prevented many mistakes. Many additional constraints were added during the campaign.
• After some doubt, the automated scheduling was found to be very convenient.
![Page 26: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/26.jpg)
K. Fuchsberger 26TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
To Improve
• Missing: Editing of Test Plans! (A lot of Db hacks were necessary )
• Constraints should be more dynamic (Currently Java Classes)… dynamic loading for plugins?
• Old page still used for statistics and quick lookups. Missing GUI features.
![Page 27: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/27.jpg)
K. Fuchsberger 27TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Next Steps
• Database Migration (Old web page will die)• Improvements of GUI statistic features
(Summer Student)• Add test-plan editor to GUI.• Improve Scheduling Algorithm ( Michael)• Integrate other systems:
• Beam Commissioning as SignOnly tests. Migrate smoothly to partly automatized tests?
• BIC/PIC tests?• Others?
• More Automated Analysis (Powering Server)
![Page 28: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/28.jpg)
K. Fuchsberger 28TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Conclusion
• System worked very well in 2011.• Some improvements (test plan editing, more
dynamic constraints, new GUI features)• A lot to improve in analysis part!• Next Steps important:
• Db migration.• Improvement of scheduling.• Improvement of automated analysis.• Integrate more systems.
![Page 29: Acctesting Framework - Motivation, Overview and First Experience](https://reader036.fdocuments.net/reader036/viewer/2022062501/56816134550346895dd08dd4/html5/thumbnails/29.jpg)
K. Fuchsberger 29TE-MPE-TM, 2012-05-10, Acctesting – Motivation, Overview and First Experience
Thank you for your attention!