Final: 2014-02-11

Sequencing the DNA of Human Talent

Our growing understanding of competencies and their relationship to job performance.

Tina Filoromo & Bill Guest ANSI Health Care Competency Collaborative

February 11, 2014, Washington DC

1

Work Styles Abilities

Knowledge Skills

Jobs defined by Tasks … grouped Competency … the ability to apply knowledge to “do” the task

2

“Competency-Based”

If the U.S. Healthcare industry was an

independent country, it would be the 5th largest economy in the world.

U.S. GDP = $15.68 Trillion Healthcare/GDP = 17.9% 15.680 x 17.9% = 2.807

2.807/71.83 = 3.91% Global

3

Healthcare is Important

2,807,000 (millions)

CIA World Factbook, 2012 estimates

4

1. Better Care for Individuals

2. Better Health for Populations

3. Lower Per Capita Costs

5

Applicant Funnel

1500 1700 2100 3000 4000 5000

15,000

1. Auto Prescreen – Personality Factors, Knock-Out Q’s

2. Manual Prescreen – Application Review, Engagement Call

3. Test – WorkKeys® (LI, WO), Hogan® (HPI, HDS, MVPI),

SkillSurvey® reference checks, and credential checks

4. Interview – HR TA Specialist with Structured Interview

Guide (SIG) including review of tests and checks

5. DASH – Day of Action Selection Hiring, with Hiring Mgr.

6. Hire – Post-Offer Checks and Employment Physical

Recommend / Hire Ratio 1.4:1

6

Data Information

Knowledge Wisdom

Data Information Knowledge Wisdom

7

8

Challenges – Drowning in a Sea of Data

• Employers have hundreds and in many cases

thousands of job titles and job descriptions

• The O*NET has 974 occupations, >100 health care

• There is no standard competency model

• Hence, competency means different things to

different stakeholders

• O*NET Model has 277 Descriptors, 120 KSAs

• Many define jobs by the tasks … ability to “do”

• There are no common standards for job

components (groups of tasks)

9

Key Organizing Dimensions

• Job Families – We organized health care into 23 job

families (>90%). This model enables thousands of job

codes to align with over 100 O*NET occupational codes.

• Competency Families – We embraced the O*NET model

of 120 KSAs. And, we summarized the model at two levels:

(1) a harmonized list of 29 competencies for job analysis

and (2) a list of 5 comprehensive competencies.

• Tasks & Task Families – We combined task lists for O*NET

occupations included in each job family by organizing tasks

around action verbs. Then, we organized the tasks into 9

Task Families by aligning with the 2nd level in the taxonomy

of the O*NET Generalized Work Activities (GWAs).

The O*NET-SOC 2010 Taxonomy

10

11

Job Families

O*NET Data: Code, Occupation, Education, and Wages

12

Job Families

EXC - Executive

MGR - Manager

FLS - Supervisor, First-Line

PS - Physician, Specialist

PC - Physician, Primary Care

AP - Advanced Practitioner

ES - Environmental Services

NS - Nutrition Services

PCA - Patient Care Assistant

MA - Medical Assistant

RN - Registered Nurse

PT - Pharmacy Technician

PLB - Phlebotomist

REG - Patient Registration, Scheduling, Admitting

BOC - BOC / Medical Secretary

LPN - LPN / Clinic (ambulatory) Nurse

CW - Case Workers

TECH - Technicians

LAB - Laboratory Technicians

MBC - Medical Billing and Coding

THR - Therapists

PHR - Pharmacists

DTN - Dietitians and Nutritionists

Leaders

Providers

Staff

Executive

Manager

Supervisor

PS

PC

AP

ES

NS

PCA

MA

RN

PT

PLB

REG

BOC

LPN

CW

TECH

LAB

MBC

THR

PHRM

DTN

23 Job Families

These 23 job

families cover

90.1% of

employees.

13

Goal: Career Paths

“Finding Your Noble Purpose”

14

O*NET Content Model

Competencies

KSAs & Work Styles

• Knowledge – Organized sets

of principles and facts applying

in general domains

• Skills – Developed capacities

that facilitate learning or the

more rapid acquisition of

knowledge

• Abilities – Enduring attributes

of the individual that influence

performance

• Work Styles – Personal

characteristics that can affect

how well someone performs a

job

KSAOs • 33 Knowledge Areas

• 35 Skills

• 52 Abilities

• 16 Work Styles

120 KSAs

USDOL Allied Health Competency Model

Job Specific

Competencies

Industry

Competencies

Foundational

Competencies

15

Knowledge

Skills &

Abilities

16

Analysis of Competency Data

• Large validation studies are inherently more reliable

than small local studies.

• Local studies can be interpreted within the context of

large studies.

• Align and analyze the data from the local job analysis

with the O*NET competency data.

Note: At this point – 7 job families done – 10 to go

Insight

17

Pareto Chart

Top 20 of 120 KSAs

18

19

20

21

98.3% of

Local KSAs

Aligned to

Green &

Yellow

O*NET KSAs

22

23

11Knowlege areas, 18 Skills, and 22 Abilities – 51 KSAs – are important across all Staff jobs.

24

25 Tasks – Linking (examples) – Individual Ratings – SME Consensus

26

Note: 22 of 31 moves, 71%, were Green to Green

27

28

29

30

31

32

Combining Tasks – Technicians

19 Technicians and 5 Lab Technicians

33 10 Codes with 12 Adjust + 2 Modify = 1 Task Statement

34

Next Steps

• Complete Job Analysis on next 7 job families (GVSU students)

• Collect longitudinal validation data on assessments SIGs

• Expand use of Hogan® personality assessments

• Explore use of ETS FACETS® personality tools (prescreen)

• Pilot career coaching and develop clear career pathways

• Pilot Prophecy® assessments for Nursing selection

• Improve the USDOL model with ANSI, HPN, H2P, and NN2

• Establish the Center for Health Care Competency Validation

• Improve the Value Logic of Competency Models with ANSI

35

What implications might this data

have on a next generation model?

ANSI

H2P

36

Diagram from: Making a Market for

Competency-Based Credentials

by CSW.

Center for Health Care

Competency Validation

37

The Value Logic of

Investing in Health Care

Competency Collaborative

38 Charlie Rose interview of Jeff Bezos on CBS 60 Minutes, December 2013

39

Appendices

40

41 National Collaboration Partners and Events

42-43 Value Logic, Illustration of Financial Returns

44-49 Validation Strategy, FFM, O*NET Data

50-56 Keys to Successful Collaboration

57-60 Moneyball, Predictive Validity, and Utility

61-66 Trinity Health Key Performance Metrics – Outcomes

67-68 OECD Adult Skills Data, Likelihood of Poor Health

69-71 Lumina Degree Qualifications Profile, DQP 2.0 Draft

72-73 ANSI HCCC Stakeholders and Talent SCM Diagram

National Collaboration

• American National Standards Institute (ANSI),

Health Care Competency Collaborative (HCCC) • May 10-11, 2012, Washington DC

• March 18-19, 2013, Novi, MI

• February 11-12, 2014, Washington DC

• Health Professions Pathways (H2P) Consortium,*

National Advisory Council (NAC) Member • October 1-2, 2012, Annapolis, MD

• April 15-16, 2013, Charleston, SC

• October 1-2, 2013, Omaha, NE

• Health Professions Network (HPN) Meetings • September 12-15, 2012, Oklahoma City, OK

• February 27-28, 2012, Las Vegas, NV

• September 19-21, 2013, Tucson, AZ (via Marianne Krismer)

*Includes: National Network of Health Care Programs in Two-Year Colleges (NN2) 41

42

43 From: Value Logic of Health Care Competency Collaborative, December 2013

Validation & Defensibility

• Contemporary I/O Psychology – Leveraged contemporary

validation methods. Reference: Alternative Validation Strategies:

Developing New and Leveraging Existing Validity Evidence, Morton

McPhail, Editor, 2007, Professional Practice Series of the Society

for Industrial and Organizational Psychology (SIOP)

• O*NET Analysis – Leveraged O*NET data, the nation’s largest

database on jobs, competencies, and occupations. Reference: An

Occupational Information System for the 21st Century: The

Development of the O*NET by Peterson, Mumford, Borman,

Jeanneret, and Fleishman, 1999 published by the American

Psychological Association (APA)

• Local Confirmatory Job Analysis – Confirming the

applicability of the O*NET to local jobs to enable point-by-point

compliance with the Uniform Guidelines on Employee Selection

Procedures (UGESP), 1978, published by EEOC

44

45

Synthetic Validation Framework

46

Locating Information & Workplace Observation

47

O*NET Work Styles (Importance) and Big 5 (FFM)

The five factors – Five Factor Model of Personality – Big 5

A summary of the factors of the Big Five and their constituent traits:

Openness to experience: (inventive/curious vs. consistent/cautious). Appreciation for art, emotion, adventure, unusual ideas, curiosity, and variety of experience. Openness reflects the degree of intellectual curiosity, creativity and a preference for novelty and variety a person has. It is also described as the extent to which a person is imaginative or independent, and depicts a personal preference for a variety of activities over a strict routine. Some disagreement remains about how to interpret the openness factor, which is sometimes called "intellect" rather than openness to experience.

Conscientiousness: (efficient/organized vs. easy-going/careless). A tendency to show self-discipline, act dutifully, and aim for achievement; planned rather than spontaneous behavior; organized, and dependable.

Extraversion: (outgoing/energetic vs. solitary/reserved). Energy, positive emotions, surgency, assertiveness, sociability and the tendency to seek stimulation in the company of others, and talkativeness.

Agreeableness: (friendly/compassionate vs. analytical/detached). A tendency to be compassionate and cooperative rather than suspicious and antagonistic towards others. It is also a measure of one's trusting and helpful nature, and whether a person is generally well tempered or not.

Neuroticism: (sensitive/nervous vs. secure/confident). The tendency to experience unpleasant emotions easily, such as anger, anxiety, depression, or vulnerability. Neuroticism also refers to the degree of emotional stability and impulse control and is sometimes referred to by its low pole, "emotional stability". From: Wikipedia – Big Five personality traits, January 24, 2014

48

Alignment with O*NET Data Categories

• Tasks – Used to create task lists for each job family

• Tools & Technology – Used to supplement task list

• Knowledge – KSA charts and competency families

• Skills – KSA charts and competency families

• Abilities – KSA charts and competency families

• Work Activities – Used to frame task families (2nd level)

• Work Context – Not used

• Job Zone – Not used

• Education – Used to group jobs into job families

• Interests – Not used

• Work Styles – KSA chart and aligned with Big 5 personality

• Work Values – Not used

• Related Occupations – Not used

• Wages & Employment Trends – Used to group job into families

49

50

Keys to Successful Collaboration

• Clear line-of-sight to the patient – always in our minds

• Two groups: those who serve patients directly, and those who

serve patients indirectly

• Senior leadership team as peers and customers of HR

• EBSP model analogous to evidence-based medicine

• Trinity Health culture work, Guiding Behaviors, prepared the way

• Clearly communicated “pursuit of excellence”

• Using assessments as data to drive better decisions and results

• Change leadership events with HR team

• Focus on better tools to increase expertise of talent acquisition

specialists, not a focus to automate jobs

The Need for Talent Excellence Organizations need the right people with the right knowledge, skills, abilities, and behaviors in the right place at the right time. Individuals need to find jobs that will fully utilize and reward the application of their knowledge, skills, abilities, and behaviors to an organization’s goals. Individuals need clarity around requirements for positions with higher rewards to use as guidance for career and education planning.

51

A “Big Idea” in Talent Excellence Improve the frequency of good selection decisions from 50% to 95%. That is, “good” based on ratings of 4 and 5 in response to: “We hired the right person for this position.”

5 = Strongly Agree Clearly on the path to become a top performer 4 = Agree Clearly capable of becoming a top performer 3 = Agree somewhat OK, but not likely to become a top performer 2 = Disagree We wish they were gone, but they’re still here 1 = Strongly Disagree Likely gone within the first 90 days

This is a 10-fold reduction in bad decisions. The defect rate falls from 50% to 5%. This is analogous to LEAN’s elimination of waste. Talent Excellence is LEAN Six-Sigma for Human Resources. *Note: Selection includes external hiring and internal moves to fill a position.

52

53

Our four core processes of

Workforce Planning, Talent

Acquisition, Talent Development,

and Performance Excellence all

work within the context of our

organizational culture.

Trinity Health - Best People / Spiritual Workplace Model

Workforce

Planning

Talent

Development

Talent

Acquisition

Future

State

Current

State

Transformation Projects:

1. Workforce Planning

2. Talent Acquisition

3. Talent Development

4. Performance Excellence

5. Job Profiles (KSAs)

6. UEM Collaboration

Culture

Performance

Excellence

54

Applicant Goals

Job Analysis

Competencies

Applicant

Test Scores

Evidence-Based Selection and Evidence-Based Medicine

55

Evidence-Based Selection Toolkit

• WorkKeys® – Locating Information & Workplace Observation

• Hogan® – HPI, HDS, and MVPI

• SkillSurvey® – Reference Check System

• Certiphi® – Education, Licensure, Certification Check System

• Interview – Structured Interview Guides with BARS

• Employment Physical & Drug Screen

• ETS FACETS® – Planned for prescreening tool

• Benchmark Partners – Exit surveys and interviews

• Position Manager® – Applicant Tracking System (ATS)

• PeopleSoft® – Human Resources Information System (HRIS)

Talent Acquisition Specialists make selection recommendations

56

57

The NCRC is Reading, Math, and Locating Information. Talent is the name of ACT’s personality test. Performance is the name of ACT’s integrity test. Talent includes Conscientiousness, Work Discipline, Customer Service and Teamwork.

Predictive Validity – Job Performance

WorkKeys

Interviews

Personality

Personality

*Note: GMA (r) varies : .58 Professional & Managerial .56 High Level Technical .51 Medium Complexity (62%) .40 Semi-skilled .23 Completely unskilled

58

3

1

5

2

6

4

R A N K

The Validity and Utility of Selection Methods in Personnel Psychology:

Practical and Theoretical Implications of 85 Years of Research Findings

Utility – Job Performance

Dollar Value of Output:

Output as a Percentage of Mean: Unskilled & Semiskilled 81% 100% 119% 38%

Skilled 68% 100% 132% 64%

Managerial & Professional 52% 100% 148% 96%

-1σ mean +1σ Variation

Example variation for -40% 100% +40% 80%

a $40,000 worker. -16,000 Mean +16,000 $32,000

59

Characteristics for Screening

• Integrity – Safety (Risk Reduction)

• Conscientiousness (General Work

Attitudes)

• Work Discipline

• Teamwork

• Customer Service Orientation

• Applied Math

• Reading for Information

• Locating Information

• Workplace Observation

• Listening for Understanding

Performance Domains

• Overall Performance

• Task Proficiency

• Effort

• Safety

• Teamwork

• Follow Rules (CWB)

• Customer Service

• Critical Thinking

• Caring and Compassion

Ultimate goal: improved

patient outcomes.

Screen Using Predictors of Performance

Predict

Performance

60

Evidence Based Selection Process

Key Performance Metrics

• Reduce First-Year Turnover

• Reduce Time-To-Fill

• Reduce Recommend / Hire Ratio

• Increase Quality of Selection (QOS)

• Increase Diversity

61

Bottom 25% Top 25% Middle 50%

Benchmark Reports: PwC Saratoga, The Advisory Board, LEAN Human Capital

62

28%

12.2%

7.3%

63

17%

Overall

Improvement

64

65

80% of Candidates recommended by OTE

were offered a job and accepted (341/426).

Bottom 25% Top 25% Middle 50%

10:1 5:1

66

Quality of Selection (QOS)

ES

NS

MA

PCA

Up

0.5

Up

0.2

Up

1.0

Flat

67

United States

3.92% Below Level 1

13.57% Level 1

17.49% At or Below 1

32.55% Level 2

68

United States

Level 1 and Below:

17.5% x 4.19 = 73 (36%)

Level 2:

32.5% x 2.43 = 79 (39%)

Levels 3, 4, and 5:

50.0% x 1.00 = 50 (25%)

69

Lumina Degree Qualifications Profile (DQP) 2.0 Matrix

70

DQP AREAS OF LEARNING

Specialized Knowledge points to learning outcomes that reflect specific areas of study—the terminology, theories, tools, literature, complex problems, and methods that characterize the field or major.

Broad, Integrative Knowledge transcends specific disciplines and involves students in ongoing, integrated ways (not only in the first two years) in the practices of core areas ranging from the sciences to the social sciences through the humanities and the arts, as well as in developing global, cultural, and democratic perspectives.

Intellectual Skills encompasses five crosscutting competencies that should transcend disciplinary boundaries: analytic inquiry, use of information resources, engaging diverse perspectives, quantitative fluency, and communication fluency.

Applied Learning focuses on what students can do with that they know, as demonstrated through their addressing unscripted problems in a variety of settings, both academic and beyond, bringing together theory and practice.

Civic Learning requires both knowledge and a commitment to action and relies on students' out-of-classroom experiences and capacity for analysis and reflection.

(Adapted from The Degree Qualifications Profile, Lumina Foundation, 2011.)

71

ANSI – Competency Model Stakeholders

Job

Industry

Foundational

Schools Curriculum

Alignment

Workforce Training and

Development

Professions Standards

Employers Sourcing, Selection,

and Development

Employers Job Descriptions

States Licensing

72

HHS HRSA, CMM,

Bureau of Health

Professions

Payers Private & Public

Community Health, Good

Jobs, etc.

Patient Compassionate,

Safe, Quality

Care

Can we improve the

performance of our

Talent Supply Chains

by working together on

competency models?

The Talent Supply Chain

73

Job

Industry

Foundational

Copyright © 2014 by Bill Guest, Metrics Reporting

Permission is hereby granted to reproduce or reprint

quotations from this work provided that such reproductions or quotations are not altered or

edited in any way and provided that an appropriate credit line and copyright notice are included.

Download more information on Talent SCM and Employer Councils at: www.ncrcadvocates.org

Talent Excellence Portions of this work contain copyrighted and trademarked materials that are used with permission.

Talxcellenz® is a trademark of Metrics Reporting, Inc. WorkKeys® and National Career Readiness Certificate® are registered trademarks of ACT.

74

Contact Information: Tom Karel, 616-685-6362, kareltl@trinity-health.org

Tina Filoromo, 248-489-6957, filoromt@trinity-health.org Bill Guest, 616-430-0828, bill.guest@metricsreporting.com

Jim Sharf, 571-274-2154, jim@jimsharf.com