Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor...
-
Upload
alexina-hillary-green -
Category
Documents
-
view
227 -
download
0
Transcript of Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor...
![Page 1: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/1.jpg)
Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment
May 1, 2012
Presented to: Staff from SCE and PG&E
Prepared by: ASW EngineeringSBW Consulting, Inc.
![Page 2: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/2.jpg)
Part Two of Today’s Presentations
SCE Contractor Training Assessment In-depth examination of SCE’s training
designs, methods and materials. Do they prepare contractor’s to succeed?
Context Rapid change in training underway, so
assessment looked at a moving target. Focused on SCE but there are wider
implications. Recommendations from the overall process
evaluation need to be considered before training is changed.
2May 1, 2012
![Page 3: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/3.jpg)
Training Big Picture
Contractors must attend training to qualify Performance gaps – a training problem?
3May 1, 2012
![Page 4: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/4.jpg)
Assessment Objectives
Effectiveness and appropriateness Do the training objectives address EUC
tasks?
4May 1, 2012
![Page 5: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/5.jpg)
Assessment Methods
Reviewed program materials (program design, training and reporting)
Interviewed key personnel (RHA, ICF) Mapped training objectives to program
requirements Assessed discrepancies
5May 1, 2012
![Page 6: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/6.jpg)
Assessment Criteria
How well the training and materials: Supported behavior change Included motivation to action Applied adult learning principles
6May 1, 2012
![Page 7: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/7.jpg)
Adult Learning Principles and Practices
Learner orientation, buy in, engagement Learner success engineering Practice, application, and interactivity Learning objectives
7May 1, 2012
![Page 8: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/8.jpg)
Effect of Teaching Styles and Average Retention Rates
8May 1, 2012
![Page 9: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/9.jpg)
Delivery Mode and Average Retention Rates
9May 1, 2012
Practice and application are crucial to training effectiveness
![Page 10: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/10.jpg)
Relating Learning Levels to Performance Outcomes
10May 1, 2012 Bloom’s taxonomy
![Page 11: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/11.jpg)
11
Required Training
Participation Workshop Basic Path Training Advanced Path Training Online Learning Center Mentoring
May 1, 2012 11
![Page 12: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/12.jpg)
Participation Workshop
2-hour session required for at least one person from each contractor – webinar or classroom
Orients contractors to goals, scope of work, marketing
Describes contractor qualifications, benefits
12May 1, 2012
![Page 13: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/13.jpg)
Participation Workshop Findings
Good pitch of program Lacks details on:
Job processing expectations (Job Reporting, EnergyPro, documentation)
Level of rigor for QA/QC
13May 1, 2012
![Page 14: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/14.jpg)
Participation Workshop Recommendations
More content on: EnergyPro requirements Exposure to job reporting
documentation and process Job reporting may change based on other
recommendations Refer to:
“Adopt Common Statewide Job Reporting” (Whole House Process Evaluation – PG&E_SCE Interim Report Workbook, Caption Tab, ID=13)
14May 1, 2012
![Page 15: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/15.jpg)
Basic Path Training
Mandatory for Basic Path program Exception if BPI Building Analyst on staff
15May 1, 2012
![Page 16: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/16.jpg)
Basic Path Training Findings
California Building Performance Contractor’s Association (CBPCA) 3-day course a good match-up between training content and program measures
Lacks: Formal objectives Adult learning principles
16May 1, 2012
![Page 17: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/17.jpg)
Basic Path Recommendations
Incorporate: Worksheets More hands-on practice Ways to demonstrate proficiency (e.g.,
asbestos identification and insulation measurement)
Formal learning objectives Adult learning principles
17May 1, 2012
![Page 18: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/18.jpg)
Advanced Path Training
EUC requires BPI Building Analyst certification
Certification can be obtained with no formal training
Certification does not guarantee EUC skill set
Dozens of third-party trainings offered
18May 1, 2012
![Page 19: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/19.jpg)
Third Party BPI Training
Quality and depth of training ranges widely
0 to 12 days Classroom, online, self-trained Insufficient practice for EUC
fieldwork (performance gaps) Loose alignment with some
measures (air sealing, weatherization)
19May 1, 2012
![Page 20: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/20.jpg)
Advanced Path Training Findings
CBPCA 12-day course provides: Solid foundation for Advanced Path work Strong combustion safety curriculum Some hands-on practice with diagnostic tools
Lacks: Formal training objectives Adult learning principles Descriptive text for photos
20May 1, 2012
![Page 21: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/21.jpg)
Advanced Path Recommendations
For CBPCA 12-day course: Add training objectives Apply adult learning principles Add descriptive text to photos
Supplement BPI training with: More hands-on practice with field testing procedures,
protocols Practice of program-specific desk work (forms, energy
modeling) Remedial training for specific performance gaps
(asbestos, air sealing, etc.)
21May 1, 2012
![Page 22: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/22.jpg)
Concerning HVAC Quality Installation (QI Program) and EUC
For minor HVAC repair work, BPI training is adequate
For major HVAC repair or replacement, then a permit is required and contractor must meet Title 24 quality standards
HVAC QI training addresses Title 24 code
22May 1, 2012
![Page 23: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/23.jpg)
Online Learning Center – A New Resource
On-demand training on: JRT Administration Energy Modeling Job Processing Combustion Safety Testing
Mandatory participation; must pass exam
23May 1, 2012
![Page 24: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/24.jpg)
Online Learning Center Findings
Well designed training Addresses some performance gap issues Incorporates adult learning features Useful for BPI certified contractors who did
not attend CBPCA 12-day course
24May 1, 2012
![Page 25: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/25.jpg)
Online Learning Center Recommendations
Record exam scores for multiple employees
Expand topics: BPI and program standards Field inspection / testing Asbestos identification Insulation and weatherization
25May 1, 2012
![Page 26: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/26.jpg)
26
Mandatory Mentoring
Required of one person from each contractor for both program levels
Three topics: Field Work Job Reporting/Job Processing Energy Pro
May 1, 2012
![Page 27: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/27.jpg)
Mentoring Findings
Not formal training per se Mentoring includes:
Observation of contractor performance Evaluation and scoring Feedback
27May 1, 2012
![Page 28: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/28.jpg)
Mentoring Recommendations
Standardize mentoring and evaluation protocols
QC firm should provide input Provide remedial training adapted to
individual needs, based on QA/QC findings Remote mentoring for Energy Pro and Job
Reporting
28May 1, 2012
![Page 29: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/29.jpg)
29
Top 10 Performance Gaps
Performance Gaps Inaccurate insulation R-value Inability to identify asbestos Failure to achieve minimum air-sealing Inaccurate wall, window, and floor measurements Improper combustion safety testing (CO readings) Failure to identify combustion appliance hazardous
conditions (e.g., gas leaks) Failure to complete Basic Path measures for
Advanced projects
May 1, 2012
![Page 30: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/30.jpg)
Top 10 Performance Gaps
Performance Gaps Inaccurate insulation R-value Inability to identify asbestos Failure to achieve minimum air-sealing Inaccurate wall, window, and floor measurements Improper combustion safety testing (CO readings) Failure to identify combustion appliance hazardous
conditions (e.g., gas leaks) Failure to complete Basic Path measures for Advanced
projects
30May 1, 2012
![Page 31: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/31.jpg)
Top 10 Performance Gaps (continued)
Performance Gaps (continued) Information does not agree across documents (JRT,
Energy Pro, forms) Inaccurate duct leakage tests
Inaccurate assessment of insulation coverage and quality
Refer to: “Performance Gaps Observed in SCE Whole House
Training” (Whole House Process Evaluation Interim Report Workbook, Caption Tab, ID=74)
31May 1, 2012
![Page 32: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/32.jpg)
Proposed Inspector-Trainer Role
Establish an internal inspector-trainer within each contractor’s staff—will be responsible for: Attending all required training Providing in-house training Ensuring quality of work
Suggestions: Collaborative effort between QA/QC (RHA) and mentors
(ICF) for training inspector-trainers Develop and provide train-the-trainer materials for
inspector-trainers
32May 1, 2012
![Page 33: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/33.jpg)
Proposed Marketing and Sales Training
Finding Little marketing and sales content in current
training materials Recommendation
Suggest formal marketing and sales training focused on contractor sales force
Classroom opportunity to practice EUC program sales skills
Opportunity to apply adult learning principles
33May 1, 2012
![Page 34: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/34.jpg)
From Interim Report - Ongoing Safety Training Findings
Safety issues are noted frequently and may be missed by contractors
34May 1, 2012
Safety Issue SCE/SCG (n=16) PG&E (n=51)Possible asbestos in ducts - noted by inspector 7 NAPossible asbestos in ducts - noted by contractor 3 2CAZ failure in Test-in 4 12CAZ failure in Test-out 6 3CAZ failure not corrected after Test-in failure 2 0Mechanical ventilation recommended or required - noted in pre-application
1 4
Mechanical ventilation recommended or required - noted in post-application
7 7
![Page 35: Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment May 1, 2012 Presented to:Staff from SCE.](https://reader036.fdocuments.net/reader036/viewer/2022062304/56649e615503460f94b5c492/html5/thumbnails/35.jpg)
From Interim Report - Ongoing Safety Training (continued)
Recommendations Provide ongoing safety training to contractors. Based on
feedback from inspectors and QC Issues to emphasize, include:
Low house air leakage Possible asbestos in ducts CAZ testing Correct installation of insulation
Refer to: “Ongoing safety training” (Whole House Process Evaluation – PG&E_SCE
Interim Report Workbook, Recommendations Tab, ID=17) Relevant To
PG&E and SCE, BIG and ICFI / RHA
35May 1, 2012