June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.
-
Upload
maximillian-chase -
Category
Documents
-
view
217 -
download
0
Transcript of June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.
![Page 1: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/1.jpg)
June 19, 2012
Evolving Your PBT ProgramThe Citrix Story
Kpayah Tamba, Citrix
![Page 2: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/2.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
About Citrix
![Page 3: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/3.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Multiple-choice Items
Scenario-based Items
Simulations
Combination of Live Labs, Simulations, Board Defense, and Scenario-based Items
At Citrix, we continue to raise the bar
Real World Skills
Live Labs
![Page 4: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/4.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Citrix Certification Program
CCIA
CCEE
CCAA
CCA
Top 3 cert tiers consist of exams with performance based items
![Page 5: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/5.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Design Approaches at each Tier
Board Defense, Live Lab, and Simulation
• CCIA
Simulation and Scenario-based
• CCEE
Simulation and Scenario-based
• CCAA
Mostly Scenario-based, with one exam containing simulation
• CCA
![Page 6: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/6.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Citrix Evolution
1997
2003
2006
2012
Multiple-choice based exams
First PBT based exam
75% of cert tracks now
includes PBT exams
First simulation
based exam
![Page 7: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/7.jpg)
How we evolved our program
![Page 8: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/8.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
• Standard multiple-choice: single response and multiple response items
Phase I: MCQ Approach
![Page 9: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/9.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Phase I: MCQ Cont’d
• Prosᵒ Easier to developᵒ Cheaper to developᵒ Easier to gradeᵒ Easier to deliver
• Consᵒ Highly vulnerable to item theftᵒ Not the best item type for assessing
performance based tasks
![Page 10: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/10.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Phase II: Live Lab Approach
• Testing using the real products in a virtual environment
![Page 11: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/11.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Phase II: Live Lab Cont’d
• Prosᵒ Test takers embraced this new form of
testingᵒ Citrix was able to measure hands-on
skillsᵒ The value (face validity) of our
certification program increased
• Consᵒ This new form of testing was not
scalableᵒ Candidates had to travel long distances
to take the examᵒ The design and delivery mechanisms
limited the number of exams that could be delivered at any given point
ᵒ The manual scoring resulted in long wait times for results
![Page 12: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/12.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Phase III: Simulation Approach
• Testing using simulated environments that look and function almost like the real products
![Page 13: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/13.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Phase III: Simulation Items Cont’d
• Prosᵒ Simulations are scalableᵒ Simulations allow for the measurement
of hands-on skillsᵒ Scoring is automated
• Consᵒ Simulations are costly to buildᵒ Simulations take a considerable
amount of time to design and developᵒ Our simulations are closed-ended;
therefore they imposed limitations on our test takers
![Page 14: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/14.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Phase IV: Board Defense and Live Lab Items Approaches
• In 2012, Citrix rolled out a six-week intensive training program geared towards certifying our top level audience
• At the end of six weeks of training (eight hours a day, five days a week), candidates are required to complete a rigorous 14-hour hands-on practicum in one sitting, where they build a virtualization solution consisting of Citrix products
• Candidates must also defend the solution they build before a panel of SME judges
![Page 15: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/15.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Phase IV: Board Defense and Live Lab Items Cont’d
• Prosᵒ The use of both live lab and simulation
items allow for Citrix to continue to measure hands-on skills at a high level, while using two different design approaches based on our need to scale
ᵒ The use of the board defense approach allows us to determine whether test takers know the “why” behind the “how”
• Consᵒ The cost to develop simulation items is
still a problemᵒ Simulations take a considerable
amount of time to design and developᵒ Live labs are difficult to scaleᵒ Live labs are difficult to scoreᵒ A uniform board defense experience is
also difficult to maintain
![Page 16: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/16.jpg)
So what’s next for us at Citrix?
![Page 17: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/17.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Next Steps
• Increase our PBT footprint
• Fully automate the scoring of our live lab items
• Complete the development of a grading tool for our board assessment
• Localize our live lab items
• Localize our board assessment exam
![Page 18: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/18.jpg)
Conclusion
![Page 19: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/19.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Conclusion
• As you embark on your next steps to expand your PBT program, ask yourself the following questions:ᵒ How will this design approach improve my program?ᵒ How do I start?
• Do I have the budget?• Do I have the skills, and if not, can I hire or contract the right talents?
ᵒ What is my end goal?ᵒ How do I get there?
![Page 20: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/20.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Q & A
![Page 21: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/21.jpg)
Work better. Live better.
![Page 22: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/22.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Group Activity• The design approach document provides:
ᵒ Examples of design approachesᵒ Characteristics of those design approachesᵒ When to use themᵒ When not to use them
• Exercise Objective:ᵒ This exercise will help us identify solutions to potential issues that one could encounter
when using a design approachᵒ We will break up into four groupsᵒ Each group will focus on an approach
• Group A – Live Environment• Group B – Board Assessments/Oral Exams• Group C – Simulating Software Using Software• Group D – Simulating Hardware Using Software
![Page 23: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/23.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Group Activity – Cont’d
• In each group, for the assigned design approach, identify solutions to the following issues:ᵒ Skills required to develop item typeᵒ Delivery challengesᵒ Psychometric challengesᵒ Other challenges
![Page 24: June 19, 2012 Evolving Your PBT Program The Citrix Story Kpayah Tamba, Citrix.](https://reader030.fdocuments.net/reader030/viewer/2022032705/56649dd05503460f94ac5716/html5/thumbnails/24.jpg)
© 2012 Citrix | Confidential – Do Not Distribute
Wrap Up
• We will regroup
• A representative from each group will share with us how his or her group plans to address the challenges associated with their assigned design approach