Talent Management Assessment Solution · Talent Management Assessment Solution: Leadership...

Talent Management Assessment Solution:Leadership Potential Report

RESEARCH GUIDE AND TECHNICAL MANUAL

Talent Management Assessment Solution:Leadership Potential ReportResearch Guide and Technical Manual

© Korn Ferry 2018. All rights reserved.

No part of this work may be copied or transferred to any other expression or form without a license from Korn Ferry.

For the sake of linguistic simplicity in this product, where the masculine form is used, the feminine form should also be understood to be included.

www.kornferry.com

Talent Management Assessment Solution: Leadership Potential Report Research Guide and Technical Manual

Item Number KFTM-01 Version 18.1a—12/2018

Contents

Section 1 Introduction ..........................................................................................................................................................1Korn Ferry Assessment Solution ................................................................................................................................1

Leadership Potential Report .........................................................................................................................................1

Intended uses...................................................................................................................................................................... 2

Purpose of this manual .................................................................................................................................................. 2

Organization of this manual ........................................................................................................................................ 3

Section 2 High potential identification .................................................................................................................... 5Business need ..................................................................................................................................................................... 5

What is leadership potential? ..................................................................................................................................... 5

Section 3 Korn Ferry’s model of leadership potential ..................................................................................... 9Overview ............................................................................................................................................................................... 9

Leadership Traits ............................................................................................................................................................... 11

Persistence ...................................................................................................................................................................... 11

Tolerance of Ambiguity ........................................................................................................................................... 12

Assertiveness ................................................................................................................................................................ 12

Optimism ........................................................................................................................................................................ 13

Learning Agility ................................................................................................................................................................14

Mental Agility ................................................................................................................................................................ 15

People Agility ...............................................................................................................................................................16

Change Agility ..............................................................................................................................................................16

Results Agility ...............................................................................................................................................................16

Situational Self-Awareness .................................................................................................................................... 17

Drivers ................................................................................................................................................................................... 17

Collaboration ................................................................................................................................................................18

Power ................................................................................................................................................................................18

Challenge ........................................................................................................................................................................19

Experience ...........................................................................................................................................................................19

Derailment Risks ...............................................................................................................................................................21

Volatile ............................................................................................................................................................................22

Micro-Managing ..........................................................................................................................................................22

Closed ..............................................................................................................................................................................23

Capacity ...............................................................................................................................................................................23

Role Preferences .............................................................................................................................................................24

Section 4 Measurement approach ...........................................................................................................................27Forced-Choice Item Response Theory ................................................................................................................27

Response distortion and faking .........................................................................................................................27

Forced-choice IRT models ....................................................................................................................................28

Administration and timing .........................................................................................................................................29

FC-IRT item administration ................................................................................................................................. 30

Other item administration .................................................................................................................................... 30

FC-IRT scoring................................................................................................................................................................. 30

Scale score reporting ....................................................................................................................................................32

Target levels and norms...............................................................................................................................................32

Section 5 Technical characteristics ..........................................................................................................................35Reliability.............................................................................................................................................................................35

Construct validity ..........................................................................................................................................................36

Criterion-related validity ............................................................................................................................................40

Prediction of Work Engagement ............................................................................................................................42

Appendix A. Frequently asked questions ............................................................................................................ 43

Appendix B. Norm descriptions .................................................................................................................................47

Appendix C. Global Personality Inventory (GPI) definitions ......................................................................65

Appendix D. GPI and Leadership Potential Report scale correlations ..................................................67

References ............................................................................................................................................................................ 69

Leadership Potential Report Research Guide and Technical Manual

© Korn Ferry 2018. All rights reserved. 1

Section 1 Introduction

Korn Ferry Assessment Solution

Korn Ferry’s online assessment solution offers an innovative process for assessing talent. Deployed on a technology platform that enables client self-service, our approach shifts the focus away from specific assessment products to solutions suitable for applications across the talent life cycle. Whether a need pertains to talent acquisition or talent management, the participant will experience a seamless assessment process.

We now have ONE online assessment solution

Which addresses TWO client objectives

Talent Acquisition

Solutions

Talent Management

Solutions

Korn Ferry Assessment Solution

• Entry Level• Graduate• Professional• Managerial/

Leadership

• Leadership Selection• Leadership

Development/Succession

• Professional Development

• High Potential

This technical manual provides psychometric-based information related to the Leadership Potential Report, which is part of one of our Talent Management assessment solutions: Potential.

Leadership Potential Report

Korn Ferry’s Talent Management solutions include world-class, science-driven self-assessments that can be combined to meet specific talent needs. Leadership potential is about what could be at some point in the future, not what is currently. By focusing on measures related to what could be, our Leadership Potential Report has been carefully conceived and empirically designed to provide critical data about people—data proven to differentiate those who have successfully advanced from those who have not advanced. Specifically, the Leadership Potential Report provides information about individuals’ Leadership Traits, Learning Agility, Drivers, Experience, and Derailment Risks. It offers critical insight for individuals and organizations to consider as they think about individuals’ likelihood of success in taking on more senior leadership roles in the future.

To create the Leadership Potential Report, we leveraged data and expertise from the combined decades of knowledge and hundreds of thousands of leadership assessments. Our model of leadership potential was developed based on rigorous analysis using a combination of quantitative, qualitative, and market-based data, sourced from both Korn Ferry’s own extensive data stores and an external literature review:

• Research analyses based on a variety of Korn Ferry intellectual property, such as Leadership Experience Inventory (LEI), Korn Ferry Four Dimensional Assessments, TalentView® of Leadership Transitions, viaEDGE® assessment, and the Korn Ferry Assessment of Leadership Potential (KFALP).


© Korn Ferry 2018. All rights reserved.2

• Extensive review of the scientific literature on high potential identification and leadership pipeline.

• Expert input representing decades of work in leadership research and development.

• Client input.

Intended uses

At Korn Ferry, we are committed to offering science-based and experience-tested assessments that support the success of our clients. The Leadership Potential Report is intended to assist organizations as they think broadly about the long-term leadership potential of their internal talent. It helps organizations identify talented people who have the characteristics needed to develop the competencies and gain the experience to succeed in future leadership roles. Specifically, the Leadership Potential Report:

• Provides organizations with the ability to objectively and accurately identify people with high leadership potential.

• Gives a standardized and objective view of a person’s leadership potential.

• Helps organizations invest in the right talent and target the right areas for development.

• Accurately identifies high potentials using characteristics proven by research to be related to advancement in leadership roles.

The Leadership Potential Report does not address immediate readiness to advance into a specific, next-level job, nor does it report fit with a particular role today. It is designed to answer the question, “Who has the potential to take on higher-level, bigger leadership roles in the future?”

In addition, the Leadership Potential Report was not developed or intended for use as a stand-alone screening tool for potential, but rather as a supplement to all available indicators of potential. It is intended to assist with careful consideration and identification of high potentials, using the full range of information available to clients and as part of a complete client process.

If the Leadership Potential Report is used to supplement other data for the selection of persons into specific jobs, the target roles should be feeder roles that are part of a leadership career progression. The progression should be designed such that, selected persons are expected within a reasonable time to advance to higher leadership roles of greater responsibility. Use of the report for this purpose assumes that the selection is for Professional/Individual Contributor, First Level Leader, or Mid-Level Leader roles. Once again, the Leadership Potential Report is best used in conjunction with all other data and information that clients may have relevant to the potential of talent pools (education, specific domain knowledge, etc.).

The Leadership Potential Report is not designed to support executive level leader selection. Once a leader has attained an executive role, evaluating their potential for advancement to the C-suite requires a more individualized level of granularity than is provided by the report. In addition, at this level, talent reviews should seek to include a view of both potential and readiness for the target roles.

Purpose of this manual

This manual is designed as a technical reference to help deepen your understanding of the research behind the Leadership Potential Report.

You can refer to this manual for a variety of purposes:

• To build your knowledge regarding the research on high potential identification.



• To enhance your understanding of the research foundation of the Potential solution.

• To review some key findings from psychometric analyses.

• To find answers to some frequently asked questions.

Organization of this manual

Relevant background and technical information for the Leadership Potential Report is presented in the following sections of the manual:

• Section 2 provides an overview of high potential identification, including the business need for this critical talent management practice and how leadership potential has been defined and tackled by organizations.

• Section 3 describes Korn Ferry’s model of leadership potential. After introducing the model, we describe each scale appearing in the Leadership Potential Report in more depth. We summarize previous research on each and articulate how they are relevant to high potential identification.

• Section 4 summarizes the measurement methods utilized in preparing the Leadership Potential Report, which includes Forced-Choice Item Response Theory (Brown, 2016; Brown & Maydeu-Olivares, 2011). In addition, we briefly outline the assessment experience and describe the nature of the scores appearing in the Leadership Potential Report.

• Section 5 presents psychometric-based information, including the results of analyses on reliability and validity. These results demonstrate that the Leadership Potential Report is highly robust, with reliable and valid assessment that meets professional standards.

• Supplemental information related to the analyses are included in the appendices. Appendix A addresses frequently asked questions.



Section 2 High potential identification

This section offers an overview of high potential identification, addressing several questions: Why are organizations concerned about high potential identification? What is leadership potential? What characteristics of individuals are related to leadership potential?

Business need

Today, organizations face a unique and unprecedented set of challenges and prospects. The pace of market change, speed of innovation, global dynamics, and changing demographics generate many opportunities to both create and extract value, but it is often more difficult to locate those opportunities and act upon them. Thus, how do companies compete in this increasingly complex and volatile environment? One of the central differentiators for companies is a strong human capital foundation: the right leaders in the right places.

To succeed in driving business strategy, it is imperative for companies to have a future-focused talent strategy. Organizations need to develop and sustain a pipeline of the right leaders, with the right abilities, in the right roles, and at the right times to ensure a sustainable competitive advantage. The idea of identifying and managing high potential talent has become increasingly essential for organizations.

Most organizations have recognized the need for and have implemented a formal process to identify and assess high potential talent (Church & Rotolo, 2013; Silzer & Church, 2009). The construct of leadership potential, as used by many organizations, refers to the possibility that individuals have the qualities (e.g., motivation, skills, abilities, experiences, and characteristics) to advance in their careers and perform effectively in future roles. It implies further growth and development to reach some desired end state. Organizations who can identify individuals with high potential are seen as having a competitive advantage (Silzer & Borman, 2017).

Companies that do have high potential identification programs frequently select individuals based on factors not necessarily related to potential, such as personal experience with the person, performance review ratings, and past performance results (Church & Rotolo, 2013; Church, Rotolo, Ginther, & Levine, 2015; Slan-Jerusalim & Hausdorf, 2007; Pepermans, Vloeberghs, & Perkisas, 2003). In addition, Martin and Schmidt (2010) indicated that based on their research on leadership transitions, nearly 40% of internal job moves made by people identified by their companies as “high potentials” end in failure.

What is leadership potential?

When comparing the way in which 13 major organizations defined potential, Karaevli and Hall (2003) found that every organization had a different definition of the construct. Silzer and Church (2009) formed several common potential definitions based on the results of a survey of organizations, finding that 35% defined potential in terms of moving into a top leadership role, 25% defined it as having the potential to take on broader responsibilities and leadership duties, 10% viewed potential as having a history of being a high performer, and 25% of organizations surveyed defined potential as having high likelihood of successfully moving up two levels from the current level. With so many different perspectives and definitions of potential in use, it may be most useful to start by defining what potential is not.

Potential is not the same as current job performance. Current performance is directly visible, but potential is a prediction about the future. Not all high performers are high potentials. Research suggests that only about 30% of high performers should be classified as high potentials (Corporate



Leadership Council, 2005). Although previous performance can be a useful piece of information in the identification of high potential individuals, it should not be (though, often is) confused with potential (Corporate Leadership Council, 2005; Slan-Jerusalim & Hausdorf, 2007; Pepermans et al., 2003).

The distinction between performance and potential is highlighted by the fact that the factors required for successful performance changes from one organizational level to the next. The Charan, Drotter, and Noel (2011) six-passage model is often used to describe the leadership requirements throughout the various organizational levels within a company. This “Pipeline Model of Leadership Development” defines the crucial skills for successful management transitions from the very bottom of an organization (managing oneself) to the very top (managing the enterprise). Each of the six management transitions in this model, illustrated in Figure 1, involves a major change in job requirements, leadership demands, skills, how time is spent, and work values.

Figure 1. The changing requirements of leadership

INDIVIDUALCONTRIBUTOR

FRONT-LINEMANAGER

MANAGER OFMANAGERS

BUSINESSUNIT LEADER

SENIOREXECUTIVE

CHIEFEXECUTIVE

Short–term Long–term

Limited stakeholders Multiple stakeholders

Manage tasks Manage portfolio

Get the job done Maximize shareholder value

Transactional Transformational

TECHNICAL SKILLS LEADERSHIP AND MANAGEMENT SKILLS STRATEGIC BUSINESS ACUMEN

Adapted from The Leadership Pipeline: How to Build the Leadership Powered Company, by R. Charan, S. Drotter, and J. Noel, 2011.

When advancing to leadership positions of greater responsibility, leadership roles increase in their challenge, breadth, and complexity. As leaders advance, they must reallocate their focus so that they can help others to perform effectively. They must learn to value the work of leadership and believe that making time for others, planning, coordinating, and coaching are imperative in their new responsibility. Strong performance at one level does not necessarily indicate the capability to successfully progress to more complex future leadership roles.

Potential is also not a person’s readiness at present to move into an immediate next-level position or promotion. Individuals who are ready to move into the next role in a year or two should already have acquired most of the knowledge and skills needed to perform those roles effectively. Potential is a more long-term concept. The identification of individuals with leadership potential involves evaluating the probability that, with additional experience and development, they will be successful in future, unspecified leadership roles (Silzer & Borman, 2017).

Another important question frequently asked when discussing the identification of high potential individuals is “potential for what?” Everyone has potential, but to what end (Silzer & Church, 2009). High potential employees are typically selected earlier on for a certain level or type of role in the



future. Some organizations have sub-pools of their high potentials for future marketing executives or engineering executives (Dowell, 2010), but over time both the individual and the role itself are likely to evolve into something a little different, and thus more general positions or leadership levels tend to be better targets for high potentials rather than very specific roles. Some of the common job band levels used by organizations include global leaders/senior executives, mid-management/technical-functional, and high-value individual contributors or high-performing professionals (Silzer & Church, 2009). The construct of leadership potential, as used by many organizations, refers to the possibility that individuals have the qualities (e.g., motivation, skills, abilities, experiences, and characteristics) to effectively perform and advance in their careers. It implies further growth and development to reach some desired end state.

As organizations identify high potential employees and invest significant amounts of time and money to develop these individuals, including providing them with the most challenging assignments, accurate identification is critical.

Several researchers have worked to identify individual attributes that are related to long-term potential (Corporate Leadership Council, 2005; Lombardo & Eichinger, 2000; McCall, Lombardo, & Morrison, 1988; Peterson & Erdahl, 2007; Silzer & Church, 2009). Based on an extensive literature review of nine external high potential models from consulting firms and two corporate surveys, Silzer and Church (2009) identified seven characteristics that are commonly viewed as indicators of high potential employees:

• Cognitive skills include conceptual or strategic thinking, breadth of thinking, cognitive ability, and dealing with ambiguity.

• Personality variables include interpersonal skills, dominance, stability, resilience, and maturity.

• Learning ability includes adaptability, learning orientation, learning agility, and openness to feedback.

• Leadership skills include developing others, leading and managing others, and influencing and inspiring.

• Motivation variables include energy, engagement, drive for advancement, career drive, interests, career aspirations, results orientation, and risk-taking.

• Performance record includes leadership experiences and performance track record.

• Knowledge and values include cultural fit and technical/functional skills and knowledge.

They further refined these into three types of dimensions of potential, which could each be measured at different stages during an individual’s career (Silzer, Church, Rotolo, & Scott, 2016):

• Foundational dimensions include more stable attributes, such as cognitive skills and personality, which could be measured even at very early career stages.

• Growth dimensions involve abilities to learn, as well as motivational variables, which may appear more clearly in certain job contexts over others.

• Career dimensions include the more level/function-specific predictors, such as performance records, leadership skills, and experiences, which should be measured and normed against the different stages during career progression.

Note that these dimensions place greater emphasis on “who people are” than “what they can do” now. Evaluating individuals’ potential should take into consideration a broad array of underlying attributes, rather than focusing narrowly on the competencies and skills needed to effectively perform the next role (Reynolds, McCauley, & Tsacoumis, in press).



Section 3 Korn Ferry’s model of leadership potential

In this section, we introduce Korn Ferry’s model of leadership potential. After providing a brief overview, we delve more deeply into each scale that appears in the Leadership Potential Report. We describe each underlying construct and summarize prior research supporting its use in high potential identification.

Overview

Korn Ferry defines leadership potential as “the capacity and interest to develop the qualities required for effective performance in significantly more challenging leadership roles.” Our model of leadership potential was formed on the basis of previous findings and science, including broad types of dimensions found in the literature.

Definitions of the attributes included in the Leadership Potential Report are shown in Table 1. These traits, drivers, and experiences reflect leadership potential and are related to advancement in the leadership pipeline.

Table 1. Korn Ferry model of leadership potential

Leadership Traits

Traits are personality characteristics that exert a notable influence on behavior. Leaders possess personality traits that make them naturally inclined to lead.

Persistence: A tendency toward passionate and steadfast pursuit of personally valued long-term or lifetime goals, despite obstacles, discouragement, or distraction.

Tolerance of Ambiguity: Comfort with uncertain, vague, or contradictory information that prevents a clear understanding or direction.

Assertiveness: The degree to which a person enjoys taking charge and directing others.

Optimism: The degree to which a person tends to disregard disappointment, is satisfied with who they are, and expects the future to be bright.

Learning Agility

Learning Agility is the willingness and ability to learn from experience, and subsequently apply that learning to perform successfully under new or first-time conditions (Lombardo & Eichinger, 2000).

Mental Agility: Mental Agility is an individual’s tendency to be inquisitive and approach problems in novel ways.

People Agility: People Agility involves skill in reading others and applying the insights gained when working with others.

Change Agility: Change Agility involves embracing change and taking well-reasoned risks in the face of that change to promote new possibilities and to take ideas from vision to reality.

Results Agility: Results Agility refers to an individual’s motivation to deliver outstanding results in new and tough situations.

Situational Self-Awareness: Situational Self-Awareness involves an individual’s ability to regulate emotions, accept circumstances, live in the moment, and reserve judgment.

(table continued)



Table 1. Korn Ferry model of leadership potential (continued)

Drivers

Drivers are the preferences, values, and motivations that influence a person’s career aspirations.

Collaboration: A preference for work-related interdependence, group decision making, and pursuing shared goals.

Power: Motivated to seek influence, recognition, and increasing levels of responsibility.

Challenge: Motivated by achievement in the face of tough obstacles.

Experience

Experiences are the roles and assignments comprising a person’s career history. They sum up major work-related events and accomplishments, highlighting what an individual has had the opportunity to do and learn. Successful leaders have a foundation of experiences that enable the acquisition of new skills.

Perspective: Perspective is gained by leading across a variety of settings.

Key Challenges: Leaders gain specific key experience when they confront seminal leadership challenges.

Derailment Risks

Promising and successful leaders and professionals sometimes lose momentum in their careers. Rather than steadily progressing, they may stall, stumble, or drift off course. Derailment Risks measures use profiles or configurations of scores on traits to highlight a propensity to behave in a way that may be problematic in the future.

Volatile: The propensity to express emotions strongly and unpredictably, without apparent concern for the impact on others.

Micro-Managing: The lack of ability to trust others to do their work.

Closed: The propensity for being dogmatic or unable to adapt to different perspectives.

Capacity

Aptitude for logic and reasoning.

Problem Solving: The ability to spot trends and patterns and draw correct conclusions from confusing or ambiguous data.

Role PreferencesPreference for the work of roles requiring versatility and achievement through others vs. professional mastery and expertise.



Next, we define and describe each attribute included in the Leadership Potential Report. Relevant literature is summarized to provide deeper descriptions of each of these characteristics. This review both underscores the importance of these attributes when evaluating potential and provides descriptive information that may be useful in understanding and interpreting the Leadership Potential Report.

Leadership Traits

The more an individual’s traits align with the traits that are characteristic of successful leaders, the greater the potential for future success at higher organizational levels. Traits factor heavily into questions of leadership potential because personality profiles look substantially different at each progressive level of management (Crandell, Hazucha, & Orr, 2014). The Leadership Potential Report includes leaders’ scores on four traits: Persistence, Tolerance of Ambiguity, Assertiveness, and Optimism.

Persistence

Persistence refers to a tendency toward passionate and steadfast pursuit of personally valued long-term or lifetime goals or values, despite obstacles, discouragement, or distraction. High scorers tend to push through adversity and tend not to give up on difficult tasks and pursuits. They are typically characterized as resilient and as having stamina and long-term or stable focus. Low scorers are more likely to change course when faced with adversity, while putting emphasis on emergent opportunities and short-term pursuits and accomplishments. Persistence has reference to long-term goal or value perseverance, resilience to adversity, and is not primarily maintained by short-term periodic and ongoing work-related feedback from others or from comparison with easily defined standards of excellence.

Duckworth, Peterson, Matthews, and Kelly (2007) explain that Persistence as a construct has arguably one of the longest histories in all of psychology and particularly in the “psychology of achievement.” Several early researchers, going back as far as the late 19th century, were interested in variables that separated similar and even similarly gifted individuals into levels of achievement. Many found that persistence, perseverance, and resilience were often key differentiating traits among individuals who otherwise had similar ability levels or similar IQ (Terman & Oden, 1947; Howe, 1999; as noted in Duckworth et al., 2007). Simonton (1994) concludes that Persistence, or “grit,” is among the more certain and consistent variables that high-impact and notable historical figures most often have in common. Persistence is typically found to be generally uncorrelated with IQ levels, and its incremental utility (over IQ and aptitude) for predicting life and occupational outcomes seems well established (Duckworth et al., 2007; Ackerman & Heggestad, 1997; Moutafi, Furnham, & Paltiel, 2005; Eskreis-Winkler, Shulman, Beal, & Duckworth, 2014). In fact, its utility in predicting success is sometimes seen as the cornerstone for understanding the differential and additive utility of natural ability vs. disposition-related variables in understanding life’s outcomes—including work-related outcomes (Ericsson & Charness, 1994). High Persistence scores are associated with increased emotional stability, increased standardized test scores, achievement motivation, educational attainment, educational performance, employment retention, and retention in challenging educational programs—including highly selective military training programs (Duckworth et al., 2007; Eskreis-Winkler et al., 2014).

Persistence-like constructs are also associated with increased levels of EQ, learning agility, strategic vision, adaptability, motivation to lead, and stakeholder sensitivity among leaders or potential leaders in organizations (Dries & Pepermans, 2012). Persistence has also been positively associated with CEO and entrepreneurial success (Baum & Locke, 2004). CEOs having higher levels of Persistence-like traits tend to be more resourceful and confident. They are more effective at communicating, setting, and reaching goals, as well as growing businesses (Baum & Locke, 2004). Persistence may be characterized as a component or expression of work-related “passion” (Houlfort, Philippe, Vallerand, & Menard, 2014) which, when associated with other socio-emotional adaptive states, is positively predictive of



increased enthusiasm, discretionary effort, positive work-related relationships, positive organizational outcomes, work satisfaction, and resilience to burnout (Cardon, Wincent, Singh, & Drnovsek, 2009; Cardon, Zietsma, Saparito, Matherne, & Davis, 2005; Liu, Chen, & Yao, 2011; Philippe, Vallerand, Houlfort, Lavigne, & Donahue, 2010).

Tolerance of Ambiguity

A comfort with uncertainty and a willingness to make decisions and plans in the face of incomplete information are hallmarks of high scorers on measures of Tolerance of Ambiguity. Tolerance of Ambiguity is a common and critical component of measures used in executive selection, development, and succession contexts (Lewis & Ream, 2012). Although the strength of association may be moderated by the nature of job roles and contexts, high Tolerance of Ambiguity among executives has been almost unilaterally associated with positive individual- and company-level outcomes (Yukl & Mashud, 2010). Business climate and organizational functioning characterized by ambiguity and uncertainty has repeatedly been characterized as “the new normal” (Cone, 2013), and management professionals and managerial scientists include Tolerance of Ambiguity among the top characteristics of successful executive leaders into the foreseeable future (Gratton & Erickson, 2007; Gratton, 2010). High Tolerance of Ambiguity is markedly associated with innovation and an entrepreneurial orientation to vocational pursuits, whether within or without organizational contexts. In research examining linkages between personality and career success, constructs related to Tolerance of Ambiguity have been related to both subjective self-perceptions of success and income (Teodorescu, Furnham, & MacRae, 2017). High scorers on measures of Tolerance of Ambiguity are more likely to seek and value diverse feedback, experiment, seek opportunities for innovation, and avoid micro-managing (Kirschkamp, 2007). For medical organizations, Tolerance of Ambiguity has been called a key indicator differentiating between physicians who can and cannot successfully make the difficult transition from clinical to executive management functions (Sherrill, 2001). Interestingly, high scorers on measures of Tolerance of Ambiguity do not eschew data or avoid seeking information by which planning and executing decisions can be guided. Rather, an effective executive with an ambiguity tolerant disposition typically has a more adaptive and nimble sense of when a critical mass of key information has been gathered, and they proceed without problematic anxiety in cases where others may not when faced with information that seems inadequate or incomplete. Brainstorming to fill in data gaps, pragmatism, and contingency plans are usually key accessories for effective and highly ambiguity-tolerant executives (Strosaker, 2010).

Assertiveness

Empirical findings show Assertiveness to be a key component of leadership emergence and potential as well as results-drive and achievement orientation (Dries & Pepermans, 2012). Assertiveness measures whether people are inclined to proactively assume wide responsibility, take charge, and lead others. A notably assertive individual is convinced that she/he should be in charge, and that both individual and group outcomes will be optimized when she/he is granted group-level decision-making discretion, leadership status, authority to delegate, and authority to set or heavily influence organizational objectives. As such, high Assertiveness might be characterized, at least in part, as self-efficacy for leadership in general (Amos & Klimoski, 2014). High Assertiveness scorers may also be seen as confident, aggressive, and decisive, while low scorers are likely perceived as tentative, passive, reserved, or indecisive and more comfortable deferring to and following the lead of other individuals or groups. Low scorers may also have and attain leadership roles, but this is far more likely when leadership status has been formally assigned and is associated with known and explicated relative managerial rank and job title. High scorers on Assertiveness-like measures, on the other hand, will take charge because they feel it will benefit organizational members and collective pursuits whether or not they were told or were granted clearance to assume responsibility as such. In short, high Assertiveness individuals are, to some extent, in charge because they have decided they are in charge, and not



necessarily because somebody else, with or without authority, has told them that they are in charge. Their leadership status and effective leadership status often is or at least begins as a de facto more than a de jure leadership status.

In the extant Big Five personality literature, a construct similar to Assertiveness is sometimes conceptualized as a component of the factor of Extraversion, and is often called Dominance (e.g., Costa & McCrae, 1992; Depue & Collins, 1999). Ones, Dilchert, Viswesvaran, and Judge (2007), however, in a comprehensive meta-analytic review, show marked differential predictive utility for these two components of Extraversion—Sociability and Dominance, particularly for managerial professionals. They find the impact of Dominance on managerial performance is positive and notably different and larger than the impact of Sociability. Judge, Bono, Ilies, and Gerhardt (2002) similarly found Sociability and Dominance having separate effects on leadership. Others have conceptualized and supported Assertiveness-like constructs as belonging to higher-order factors removed from Sociability or other social-behavior-related measures (e.g., Dries & Pepermans, 2012; Northouse, 1997; Mann, 1959; Stogdill, 1948; Hogan, 1983; Wiggins, 1996). Hogan (1983) and others (e.g., King & Figueredo, 1997) in empirically-based higher-order personality structures separate Dominance from Extraversion or Sociability, concluding that the latter is better dubbed “Surgency”—having reference to general positive mood and sociability, whereas, Dominance emerges as its own factor with primary reference to confidence, independence, and aversion to submissiveness or deference. Some researchers argue Assertiveness and social variables are clearly associated, but not necessarily conceptualized as sub-components of a single common latent factor (Dries & Pepermans, 2012). Yet others (e.g., McCrae & Costa, 1987) assert that Sociability is not best combined with Assertiveness in an Extraversion factor, but that Sociability belongs with emotional and affective variables.

Assertiveness predicts both self and other rating of Sociability, as well as “competency” domains such as creativity, analytical thinking, and problem solving (e.g., Anderson & Kilduff, 2009). Interestingly, Assertiveness seems to affect others’ perceptions of competence in various leadership domains incrementally in models also containing scores of actual competence. As such, Anderson and Kilduff (2009), among others, show that high Assertiveness leaders typically instill trust and confidence in others in ways that are not always directly linked to rationality, truth, or more objective measures of actual leadership status or skill. Increased and even very high scores on Assertiveness-like measures among CEOs are positively associated with company innovation and company patent counts, and the effect is notably stronger for CEOs operating in highly competitive markets (Galasso & Simcoe, 2011). Assertiveness, however, can sometimes be associated with lack of receptivity, micro-managing, and/or need for control in ways that create challenges for team performance, particularly when high Assertiveness scores are present in individuals having low scores on affiliation-type measures or measures of EQ and/or positive affect (Driskell & Salas, 1992).

Optimism

Leaders who are high in Optimism expect the future to be bright, are likely to disregard disappointment, and are satisfied with themselves. Optimists are likely to be warm and slightly dominant interpersonally and tend to handle relationships successfully (Carver & Scheier, 2014).

Optimism is a compound trait, related to multiple factors in the Five-Factor model of personality (Hough & Ones, 2001). Originally thought to be a blend of Extraversion and Emotional Stability (Hough & Ones, 2001), Optimism has consistently been found to have moderate to strong relationships with each (Carver & Scheier, 2014; Kam & Meyer, 2012). Optimism has been shown to have somewhat smaller associations with Agreeableness and Conscientiousness (Sharpe, Martin, & Roth, 2011). Optimism also has been conceptualized as a key component of mixed models of Emotional Intelligence (Livingstone & Day, 2005).



Although Optimism is a facet of personality, it also is cognitive in nature because it involves expectations for the future. Through these expectancies, Optimism is also connected to motivation and self-regulation (Carver & Scheier, 2014). Positive expectancies are associated with more persistent and effective pursuit of goals (Segerstrom, 2007). Consequently, individuals high in Optimism appear to be highly effective at investing effort in complex situations, effectively increasing their engagement for higher-priority goals and under favorable circumstances (Carver & Scheier, 2014). Leaders who are confident about the future will continue to invest effort; those who are doubtful about eventual success are likely to disengage (Carver & Scheier, 2014).

Optimism has been associated with future success in a variety of contexts. For example, one study found Optimism measured during first semester in law school predicted higher salaries a decade later after controlling for hours worked (Segerstrom, 2007). Personality traits that contribute to Optimism have been associated with leadership emergence and effectiveness. For example, in one meta-analysis, Extraversion and Emotional Stability were robustly related to leadership in business settings; Emotional Stability and Conscientiousness were robustly related to leadership in military and government settings (Judge, Bono et al., 2002). Composites of Emotional Intelligence that include Optimism are positively related to job and life satisfaction, accounting for incremental validity in these outcomes beyond the Big Five personality traits (Livingstone & Day, 2005).

Learning Agility

Learning and skill development play an important role in an individual’s long-term effectiveness and career success (Silzer & Church, 2009; Tannenbaum, 1997). The most effective way to assess a person’s potential to learn from experience is by measuring learning agility. Learning Agility is defined as the willingness and ability to learn from experience, and subsequently apply that learning to perform successfully under new or first-time conditions (Lombardo & Eichinger, 2000). Learning agile individuals are nimble and adaptable in changing environments; they are key players who fill the leadership bench. Their ability to learn from experiences and take on novel challenges sets them apart as high potentials, as evidenced by their speedy career ascent (Dai, Tang, & Feil, 2014; Dai, De Meuse, & Tang, 2013). Nearly 25% of the Fortune 100 assess Learning Agility as one component of potential.

Learning Agility is especially crucial during job transitions—such as a promotion—when an individual invariably faces new and unfamiliar situations. Instead of automatically defaulting to favorite past solutions or problem-solving tactics, learning agile leaders apply fresh and varied approaches, ideas, solutions, and techniques to solve those new, tough problems. In short, learning agile leaders find new ways to successfully navigate unknown and unforeseen challenges.

Learning Agility is widely seen as a multidimensional concept (De Meuse, Dai, Swisher, Eichinger, & Lombardo, 2012). This view is supported by factor analyses with both multi-rater and self-report measures (Korn Ferry, 2013). We assess five Learning Agility dimensions: Mental Agility, People Agility, Results Agility, Change Agility, and Situational Self-Awareness. Each aspect is associated with others’ (e.g., bosses, human resource professionals) ratings of potential and propensity to stay out of trouble, i.e., unlikely to derail (Lombardo & Eichinger, 2000).

The ROI for organizations and leaders is clear. Research shows that learning agile leaders learn more, are rated more competent, are seen by supervisors as performing better, are recognized as having the most potential for advancement, get promoted faster and more often than their peers, and outperform their peers after a promotion (Dai et al., 2013; Dragoni, Tesluk, & Oh, 2009; Dries, Vantilborgh, & Pepermans, 2012; Laxson, 2018; Lombardo & Eichinger, 2000). Learning Agility also accounts for incremental validity in predicting performance above and beyond intelligence and personality (De Meuse, Dai, & Hallenbeck, 2010). A summary of published and unpublished studies estimated that



the average (uncorrected) correlation between Learning Agility and Potential is .47 (De Meuse, 2018). This large relationship highlights the value of including Learning Agility in assessments utilized for high potential identification.

Korn Ferry has extensive research for describing and measuring Learning Agility. This includes observable competencies and a set of related traits. Korn Ferry research found that highly learning agile people earn promotion much more quickly (Dai et al., 2014; Dai et al., 2013). After grouping individuals by low, moderate, and high Learning Agility scores, our analysis found that managers with high Learning Agility were twice as likely to be promoted over a 10-year period as those with low Learning Agility (see Figure 2).

Figure 2. Likelihood of promotions managers were to receive over 10 years

Low Learning Agility

Moderate Learning Agility

High Learning Agility

0 0.5 1

Odds of promotions

1.5 2 2.5

Mental Agility

Mental Agility is an individual’s tendency to be inquisitive and approach problems in novel ways. Individuals with high Mental Agility enthusiastically approach complex issues, new challenges, and unfamiliar situations with broad curiosity. They seek out, explore, and investigate information to develop a broad perspective. In contrast, individuals scoring lower on Mental Agility tend to focus on information that is readily apparent, take a narrow perspective, and favor existing understanding of and solutions to problems.

Mental Agility and related constructs are associated with a variety of individual-level outcomes. For example, curiosity is positively correlated with workplace learning (Reio & Wiswell, 2000) and newcomer adaptation (Harrison, Sluss, & Ashforth, 2011). It is associated with indicators of career success, including self-reported success at work and income (Teodorescu et al., 2017). High scores are related to performance after promotion, making it well suited for the assessment of leadership potential (De Meuse, Dai, & Wu, 2011).

Mental Agility is distinct from mental ability or intelligence. Intelligence involves an individual’s capacity to reason, think abstractly, and solve problems. It is related to effective performance in many roles, including leadership. However, empirical research has found relationships between measures of intelligence and learning agility are small (Bedford, 2011; Connolly, 2001). Consistent with this, the correlation between scores on the Raven’s Advanced Progressive Matrices™, a culture-free test of fluid intelligence, and the Mental Agility scale is r = .158 (p < .001, n = 37811).



People Agility

People Agility involves skill in reading others and applying the insights gained when working with others. Individuals high in People Agility understand the value of getting work done with and through people, being attuned to individuals’ needs and motivations, and having an effective influencing style. Individuals who score low on People Agility are less politically agile and prefer to let conflicts work themselves out. They are more likely to approach every situation involving people the same way.

As an element of People Agility, skill in reading others is critical to effective performance in managerial and leadership roles (Wilson, 2015). Based on a meta-analysis, the average correlation between empathetic accuracy and job performance is .20 (Elfenbein, Foo, White, Tan, & Aik, 2007). Related constructs have been linked to success in negotiations (Elfenbein et al., 2007; Galinsky, Maddux, Gilin, & White, 2008; Bazerman & Neale, 1983). Effectively influencing others is often described as part of being a charismatic or transformational leader. Being a transformational leader is progressively more important as leaders transition to more senior levels in the leadership pipeline (Charan et al., 2011), making People Agility particularly relevant for high potential identification.

Change Agility

Change Agility involves embracing change and taking well-reasoned risks in the face of that change to promote new possibilities and to take ideas from vision to reality. Individuals with high Change Agility like change, continuously explore new options, and are interested in leading change efforts. They enjoy tinkering with tasks and processes, striving for continuous improvement. In contrast, individuals low on Change Agility prefer well-established approaches, stability, and routine.

Comfort with and adaptability to change has consistent, positive effects on individual- and company-level leadership outcomes (Reeves & Deimler, 2011). Upper-level managers who are less adaptable are likely to underperform, be paid less, be mistrusted by boards, and turnover (Guay, Taylor, & Xiao, 2014). Calculated risk taking increases at higher levels of management (Delgado-Garcia, de Quevedo-Puente, & Fuente-Sabate, 2010), and adaptive risk taking is associated empirically and descriptively with confidence, prompt decision making, and promotability (Shapira, 1995). Individuals high on Change Agility are more likely to succeed in senior leadership roles.

Results Agility

Results Agility refers to an individual’s motivation to deliver outstanding results in new and tough situations. Individuals with high Results Agility are energized by novel, tough assignments. They enjoy overcoming obstacles and value accomplishing things against the odds. Embracing challenges and driving to succeed are hallmarks of high potential leaders (Ready, Conger, Hill, & Steckler, 2010). Those lower on Results Agility prefer attainable and well-understood, perhaps routine, goals.

Constructs related to Results Agility have strong, positive relationships with the performance of managers (Hough, Ones, & Viswesvaran, 1998; Dudley, Orvis, Lebiecki, & Cortina, 2006; Judge & Piccolo, 2004; Ones, Dilchert, Viswesvaran, & Judge, 2007), leadership emergence and effectiveness (Judge, Bono et al., 2002; Marinova, Moon, & Kamdar, 2013; Palanski & Carroll, 2006), and entrepreneurial performance (Collins, Hanges, & Locke, 2004). That is, individuals who are driven to deliver outstanding results and can be counted on to follow through in tough situations are likely to emerge and perform well as leaders. Among individuals who are already leaders, Results Agility-like constructs are positively related to innovation (Papadakis & Bourantas, 1998), organizational profitability (Simons & McLean Parks, 2000), and venture growth (Lee & Tsang, 2001). Overall, individuals high on Results Agility are likely to excel in leadership roles, making Results Agility a powerful tool for high potential identification.



Situational Self-Awareness

Situational Self-Awareness is the ability to be aware of present experience in a non-judgmental way, pay attention to the importance of various demands, be aware of one’s expert intuitions, and able to improvise in a dynamic environment. An emerging construct in the industrial/organizational psychology literature, Situational Self-Awareness is sometimes referred to as mindfulness and has been called a “Western adaptation to an Eastern way of thought” (Haigh, Moore, Kashdan, & Fresco, 2011). Situational Self-Awareness involves one’s ability to regulate emotions, anticipate and be proactive for change, accept circumstances, live in the moment, reserve judgment, and be aware of even subtle internal and external information. Low scorers on Situational Self-Awareness are more likely to be focused on past or future events, are less aware of their impact on situations as they occur, and are more likely to use strict and well-defined heuristics when making decisions or characterizing a situation.

Across studies and measurement instruments, Situational Self-Awareness has repeatedly shown compelling evidence of construct validity and has displayed substantive correlations with many other psychological constructs and outcomes (Haigh et al., 2011; Feldman, Hayes, Kumar, Geeson, & Laurenceau, 2007). Considerable positive relationships have been observed with positive affect, curiosity and exploration, emotional regulation, mood repair, and cognitive flexibility. Conversely, it has shown substantial negative relationships with a variety of maladaptive and problematic emotional and affective states. Specifically, increases in Situational Self-Awareness are associated with decreased anxiety, distress, depression, worry, rumination, thought suppression, avoiding experiences, and brooding (Kumar, Feldman, & Hayes, 2008; Johnson, 2007).

Situational Self-Awareness has been associated with effective strategic decision making, novelty seeking, adaptive risk taking, and awareness of key resources among key players in organizations (Langer, 2009; Nadkarni & Barr, 2008; Weick & Roberts, 1993). Interestingly, Situational Self-Awareness may also moderate the link between other psychological constructs and ratings of job performance, such that higher Situational Self-Awareness strengthens positive associations where applicable (Barrick, Parks, & Mount, 2005). Situational Self-Awareness also can provide a framework or otherwise assist in coaching and development activities that help employees manage and/or take advantage of stress, produce results while learning on the job, and mitigate derailment (Lee, 2012).

Drivers

Drivers are the preferences, values, and motivations that influence career aspirations. They provide the “will do” that creates engagement and energy for a task or role. To the extent that a person’s drivers are aligned with the role, they will be energized by it.

People with leadership potential find the role of a leader interesting and the work of leading motivating, which is crucial to being effective. Leadership becomes progressively more difficult at every level, and the demands upon time and energy increase. If the work doesn’t align to what drives them, it is unlikely that any leader will have the energy and resilience needed to thrive or even to just survive. According to Silzer and Church (2010), 90% of organizations now use an individual’s career drive as one predictor of high potential.

High potential leaders value the nature of leadership work, the opportunity to make a difference, having a positive impact on their coworkers and organization, and having greater responsibility. This is evident in the greater prevalence of goals and aspirations related to leadership at each career level.

Data collected over the past decade at Korn Ferry show that those who move up in leadership are marked by having higher career aspirations, more specific career goals, a desire to take on general management and C-suite positions, and are engaged by getting things done through others (see Table 2).



Table 2. Signals of leadership drive across management levels

Percent choosing in the top three motivators:

First Level Leader

Mid-Level Leader

Functional or Business Unit Leader Senior/Top

Influence on the direction of the organization.

38% 52% 61% 72%

Belief in the mission of the organization. 41% 41% 47% 56%

Responsibility for the performance of others and the results of the unit.

30% 42% 48% 49%

Source: Over 17,000 leaders, Career History Questionnaire (Gerstner, Hazucha, & Davies, 2012).

Three drivers are included in the Leadership Potential Report: Collaboration, Power, and Challenge.

Collaboration

Collaboration reflects preferences, values, and motives related to working with others. It involves striving to meaningfully connect with others and gain acceptance in relationships (Barrick, Stewart, & Piotrowski, 2002). Socioanalytic theorists (e.g., Hogan & Warrenfeltz, 2003) have argued that people have innate biological needs for acceptance and approval. Being connected to others, feeling a sense of relatedness, and desire for interpersonal attachment is a fundamental human motivation (Baumeister & Leary, 1995). Collaboration is associated with conformity and conflict avoidance, which are not generally typical of leaders. In fact, in one longitudinal study, McClelland and Boyatzis (1982) found that the need for affiliation was negatively related to promotion and managerial level. In today’s organizations, however, the pace of technological change, increased complexity, competitive demands, challenging economics, and risks involved in decision making have made it difficult for individuals to act alone. Leadership research increasingly emphasizes collaborative approaches to leadership effectiveness (Yammarino, Salas, Serban, Shirreffs, & Shuffler, 2012). Some scholars even suggest that leaders develop and adopt “collective identities,” which involve self-definitions based on group membership (Venus, Mao, Lanaj, & Johnson, 2012). Highly collaborative leaders are motivated by internalizing group values and norms, fulfilling social roles and obligations, and contributing to the group’s welfare. This typically cultivates trust among team members, which in turn results in increased team performance (Drescher, Korsgaard, Welpe, Picot, & Wigand, 2014). Collaborative leadership is increasingly characterized as key for innovation management. In our own data (e.g., D’Mello, 2015), we have repeatedly found Collaboration (albeit characterized as a behavior more than a motive) to be one of the most salient predictors of innovation and related outcomes.

Power

A drive for Power involves a strong desire to influence others. Individuals driven by power enjoy being held responsible for other people and broader group results. They aspire to achieve higher status and even a prestigious title or rank. They are energized by visibility and strive to gain rewards and recognition for their efforts. The essence of leadership itself is embodied in the act of influencing others, and a weak drive for power means a lack of interest in influence and impacting others (McClelland, 1965; McClelland & Burnham, 1976). In Winter’s (1987) study of US presidents, power motivation was significantly correlated with historian ratings of “greatness.” The same power motivation scores have



also been linked to ratings of certain aspects of presidential performance, as well as charisma (House, Spangler, & Woycke, 1991). After reviewing the literature, Zaccaro (2001) cited power motivation as a key and incremental predictor of leadership charisma.

Challenge

Individuals driven by Challenge prefer new and difficult projects that stretch their abilities. They tend to thrive on learning and pushing their limits to acquire new proficiencies. They are excited by the prospect of making a difference and are typically willing and eager to put forth discretionary effort in pursuit of accomplishing goals. Individuals who are high on Challenge are also typically driven by competition and the desire to win. Meta-analytic evidence links Challenge-like ratings to a variety of outcomes including income, job performance, community leadership, and sales success (Spangler, 1992). In their meta-analysis, Collins, Hanges, and Locke (2004) found that Challenge was related to choosing an entrepreneurial career as well as entrepreneurial performance. Some previous research has suggested that Challenge is more strongly linked to leadership success at lower levels of leadership, where the contributions and accomplishments of individuals are seen as more important than influence over others (McClelland & Boyatzis, 1982). Studies of higher-level managers have presented mixed results. House et al. (1991) and Deluga (1998), for example, found negative or zero relationships between Challenge and presidential performance and greatness. In contrast, Zaccaro, White, and colleagues (1997) found that Challenge was positively linked to senior leadership-potential ratings, career achievement, and organizational level in a sample of army civilian managers. Taken together, this body of research suggests Challenge is an important indicator of potential, with positive relationships to leaders’ success at most, if not all, levels of leadership.

Experience

As leaders progress through their careers, they have a series of experiences. Through these experiences, leaders gain motivation, attitudes, knowledge, skills, and other capabilities that enable them to perform effectively in future leadership roles (Campbell, 2012; Tesluk & Jacobs, 1998). Experiences, therefore, are an important foundation for moving to new, more challenging roles.

A leader who has honed skills through depth and breadth of experiences has much more bandwidth to learn everything else they must conquer to succeed when promoted to the next level. A leader who is behind the curve, who lacks one or more relevant experiences, will have to learn these lessons while they are also learning the job. This extra demand, at a time of rapid change, makes the transition risky and more likely to go awry.

Work experiences are multi-faceted, with quantitative and qualitative components that interact with each other. They vary in specificity (task, job, team, organizational, and occupational) as well as measurement mode (amount, time, density, timing, type) (Quinones, Ford, & Teachout, 1995; Tesluk & Jacobs, 1998). Consequently, experience is complex, unique to individuals, and far from universal in its nature and value.

The importance of developmental experiences was originally surfaced in a series of studies conducted by the Center for Creative Leadership. In these qualitative studies, executives were interviewed and asked to describe key events in their careers that caused the most learning. The following two questions were probed: 1) what specifically happened on the job, and 2) what did they learn from the event. Researchers interviewed 191 executives from six major corporations. Descriptions of the 616 events and 1,547 corresponding lessons were tabulated. The analyses and results are summarized in the book aptly titled The Lessons of Experience (McCall et al., 1988). These researchers observed that the most developmental experiences are challenging, stretching, and difficult.



Work experiences are challenging when existing tactics and routines are inadequate for addressing work tasks, and new ways of dealing with work situations are required (De Pater, Van Vianen, Bechtoldt, & Klehe, 2009). By presenting ambiguous situations and exposing individuals to a great diversity of organizational stimuli, challenging experiences provide critical opportunities for leaders to acquire new job knowledge and practice new skills. Stimuli that are novel, uncertain, and difficult tend to create a heightened sense of arousal within individuals. This heightened arousal will lead to a wide range of behavioral and cognitive processes, including learning (DeRue & Wellman, 2009). Challenging experiences also shape leaders’ work attitudes and motivation by uncovering gaps between individuals’ current capabilities and what is required for assignment success (McCauley, Ruderman, Ohlott, & Morrow, 1994).

Korn Ferry research has identified key career experiences that differentiate leaders. The more of these key developmental experiences a leader accumulates, the greater the possibility that the leader will be successful after promotion to the next level. Working with research partners at well-known universities, Korn Ferry has also found that experience helps leaders develop their strategic thinking skills (Dragoni, Oh, VanKatwyk, & Tesluk, 2011). In general, research has found that challenging experiences contribute to the development of a broad variety of leadership competencies including business knowledge, visioning, strategic thinking, problem solving, decision making, change management, and interpersonal skill (DeRue & Wellman, 2009; Dragoni et al., 2011; Dragoni et al., 2009). Not surprisingly, individuals who have had more challenging experiences are more likely to be perceived as having high promotion potential (De Pater et al., 2009).

Leadership Experience Inventory (LEI) data at Korn Ferry indicate that, on average, leaders at the highest levels are more likely to have had a wide range of key developmental, career-building experiences (see Figure 3). Note also that the largest jump is from Mid-Level Leader to Business Unit Leader.

Figure 3. Prevalence of key formative career experiences by management

0

10

20

30

40

50

60

70

80

90

100

Critica

l neg

otiatio

ns

Exter

nal r

elatio

ns

Inhe

rited

pro

blems a

nd ch

allen

ges

Crisis

man

agem

ent

Downtur

n an

d/or f

ailur

es

Busin

ess g

rowth

Start-

up b

usin

ess

Produc

t dev

elopm

ent

Inte

rnat

iona

l/cro

ss-c

ultu

ral

First Level Leader

Mid-Level Leader

Business Unit Leader

Senior Executive

CEO



Even within leaders at the highest levels, the impact of formative, challenging experience is observed. Among leaders evaluated by Korn Ferry for CEO roles, those who were judged most effective reported having more challenging work experiences. A higher proportion of the most effective CEO candidates had challenging experiences that were heavy in strategic and people demands (Crandell, Orr, & Urs, 2015).

To capture the Experience data for the Leadership Potential Report, we use a short, structured career walk-through that contains verifiable questions about what an individual has done to date. This is an area where we have chosen to be brief in order to keep assessment time reasonable and to allow more robust measurement in other areas. Experience is normed against the participant’s current level. Two aspects of Experience are captured:

• Perspective can be considered “horizontal experience.” It represents the leadership “variations” the participant has gained from working in different organizations, industries, roles, countries, etc. Perspective helps leaders move beyond “the way we do things here” thinking to encompass the lessons and experiences of alternative or diverse ways of approaching the role, gained through variety of perspective.

• Key Challenges captures the participant’s experience across 10 key leadership challenges that have been found to be particularly developmental. Not every leader will face each challenge during their career. Key Challenges also reflects the leader’s role in the challenge—e.g., participant, leader, or sponsor.

Derailment Risks

Derailment is the failure to achieve one’s potential. The outcomes associated with leadership derailment can be very costly on many dimensions. In addition to untold millions of dollars of direct and indirect financial costs, derailed managers can engender a negative impact at the individual, team, and organizational levels. Such leaders don’t build cohesive teams, erode the morale of coworkers, damage customer relationships, and fail to meet business objectives (Bunker, Kram, & Ting, 2002; Hughes, Ginnett, & Curphy, 2008). Researchers estimate that 30% to 50% of leaders derail (Kaiser, LeBreton, & Hogan, 2015; Lombardo & Eichinger, 1989).

The risks related to derailment go up at higher job levels: expectations are higher and consequences of failure are higher (Hogan & Hogan, 2001; Tang, Dai, & De Meuse, 2013). The potential for derailment is rated significantly higher for upper management than lower and middle management (Tang & Dai, 2013). The reasons for this increase in rated likelihood of derailment by others by level include: 1) the strengths that propel leaders to the top often have corollary weaknesses; and 2) increased demands and higher expectations yield more focused scrutiny.

Derailment Risks measures use profiles or configurations of scores on traits to highlight a propensity to behave in a way that may be problematic in stressful, ambiguous, or complex situations. That is, to handle extremely challenging tasks, leaders may act in a way that could have a negative impact on the people they work with and their teams. It cannot be overemphasized that risks are not destiny. Feedback on these risks is provided to equip leaders with the awareness needed to expand their repertoire of behaviors and avoid defaulting to potentially problematic patterns when faced with challenging circumstances.

The Leadership Potential Report provides scores on three Derailment Risks that signal a tendency to behave in ways that can cause problems for otherwise successful people.



Each is described briefly below and then discussed in more detail.

• Volatile is behaving in unexpected or detrimental ways. Effective leaders tend to be steady, even-tempered, and composed. Leaders who behave in a volatile way find it more difficult to build trust and confidence among their people.

• Micro-Managing is staying involved in too many decisions rather than passing on responsibility, doing detailed work rather than delegating it, and staying too involved with direct reports. Effective leaders allow their people to succeed through their own efforts and skills.

• Closed is being dismissive of differing perspectives. Being closed makes it more difficult to respond to the need for change or to cultivate new ideas that can improve performance of the leader or team. Effective leaders are open to the perspectives and ideas of others.

A high score on a Derailment Risk provides an early indicator that proactive development may be helpful. Evidence of Derailment Risks indicates a possible tendency toward certain behavior. Some individuals who score high on Derailment Risks will not encounter situations that trigger potentially problematic behaviors. Consequently, Derailment Risks scores are not intended to be used in employment decisions. They are intended for individual insight and development purposes only. Each Derailment Risks scale uses profiles or configurations of scores on traits or on traits and drivers. These traits and drivers are drawn from the KF4D traits and drivers scales. To a large extent, the Derailment Risks are grounded in the Big Five and related models and approaches to understanding personality. The Derailment Risks are not based on nor intended to reflect taxonomies of clinical personality disorders. That is, they are not designed to, nor is it likely they would reveal, impairments in mental health.

Volatile

Volatile reflects the extent to which individuals are likely to express emotions strongly and unpredictably, without apparent concern for the impact on others. Effective leaders tend to be steady, even-tempered, and composed. Leaders with high scores on Volatile may encounter difficulties building trust and confidence among the people with whom they work because they behave unpredictably. These individuals may be more vulnerable to the pressures of external events and internal reactions, leading to sudden displays of emotion.

Research involving similar measures of emotional volatility points to a range of undesirable outcomes stemming from the presence of this Derailment Risk. A meta-analysis by Gaddis and Foster (2015) found that managers with volatile tendencies tend to lack the trust of others at work, fail to use sound judgment to make decisions, and are generally considered to be ineffective as leaders. These authors (Gaddis & Foster, 2015) also found that emotionally volatile managers tend to have more difficulty adapting to shifting circumstances and trying new approaches, and that they may also be viewed as socially inappropriate and interpersonally challenged. Furnham, Trickey, and Hyde (2012) conducted a study to explore correlations between trait-based derailment risks and a range of occupational success factors and found that emotional volatility was significantly and negatively correlated with nearly every success factor, including managerial potential, service orientation, and reliability. Additionally, several studies have found correlations between emotional volatility and having a negative attitude at work (Gaddis & Foster, 2015; Palaiou, Zarola, & Furnham, 2016). Finally, there are also potential impacts on followers. For example, according to Spain, Harms, and Wood (n.d.), followers must use more emotional resources to cope with the emotional volatility of leaders with this Derailment Risk.

Micro-Managing

Micro-Managing is the extent to which leaders stay involved in too many decisions and perform detailed work themselves, rather than delegating. Leaders who score high on Micro-Managing may have difficulty giving up control and feel compelled to be overly directive due to their lack of ability



to trust others. Effective leaders allow their team members to succeed through their own efforts and skills.

Research has related strong scores on Micro-Managing to a number of negative outcomes. Leaders who micro-manage are often indifferent to the impact that this style can have, including negative outcomes for employees, such as cynicism, issues with self-esteem, lack of goal clarity, and diminished creativity (Alvesson & Sveningsson, 2003). Research suggests that those who tend to micro-manage may exploit rather than develop people, given their tendencies to lack confidence in others and control rather than inspire and motivate (White, 2010). This may be due to a lack of awareness of others’ motives. Additionally, individuals with a propensity to micro-manage have a low tolerance for mistakes and may even blame others for their own shortcomings or missteps (White, 2010).

Closed

Closed refers to a tendency to be dismissive of differing perspectives and rigid in thinking or approach, especially when it involves people, ideas, or solutions that seem quite different from one’s own views. Effective leaders are open to the perspectives and ideas of others. In contrast, being Closed makes it more difficult to respond to the need for change or to cultivate new ideas that can enhance performance of the leader or team and improve decision making. Rokeach (1960) described closed-minded in his classic discussion of dogmatism. More recently, it has been discussed as the need for cognitive closure (Kruglanski & Webster, 1996) and closed-minded or dogmatic cognition (Price, Ottati, Wilson, & Kim, 2015). It also touches upon intellectual humility (Leary, Diebels, Davisson, Jongman-Sereno Isherwood, Raimi, Deffler, & Hoyle, 2017), the extent to which people recognize that their beliefs may be wrong and others may be right.

Closed has implications for information seeking and processing, teamwork and social norms, as well as confirmation bias. Persons high in scores on Closed characteristics have been found to limit careful consideration of issues and act with reduced information, sometimes jumping to conclusions (Kruglanski & Freund, 1983). In information processing, active open-minded thinkers appear to gather more information and are able to make more accurate forecasts in estimation tasks (Haran, Ritov, & Mellers, 2013). In problem solving, persons with a Closed style may consider fewer alternatives (Mayseless & Kruglanski, 1987). The construct is also related to attitudes toward diversity (Kruglanski, Shah, Pierro, & Mannetti, 2002), with high Closed status related to a preference for homogeneity in teams and is also related to strong preference for clear norms. Ottati, Price, Wilson, and Sumaktoyo (2015) found that closed-minded cognitions may be a particular risk for those with high self-perceptions of expertise or in more expert roles, where a more closed-minded cognitive style supporting prior expectations is the social norm. In a work world that is increasingly diverse and dynamic, challenges faced by persons with a Closed style are likely to increase.

Capacity

Capacity refers to logic and reasoning, or cognitive ability. Research has shown that cognitive ability influences virtually every aspect of job performance and potential (Ones, Dilchert, Viswesvaran, & Salgado, 2010). It is positively related to leadership emergence and effectiveness (Judge, Colbert, & Ilies, 2004). High-performing leaders are effective analytical and conceptual thinkers. They are astute at spotting patterns or trends in data that others miss. And they solve problems with aplomb—at first individually, and then as leaders—by marshaling and focusing resources on the right challenges.

It is an individual skill and there is a subtle trap when thinking about Capacity as one moves up in leadership: a person’s role changes from being the primary problem solver to ensuring that the problem gets solved. Leaders who cannot shift out of individual problem-solving mode and into the job of coaching and mentoring others to analyze problems will struggle beyond mid-level leadership roles.



Likewise, organizations that rely on individual problem solving as their sole or even primary indicator of high leadership potential risk flooding their pipeline with people who will peak in mid-level roles because they revert to solving complex problems themselves. For this reason, it is risky to assess pure cognitive ability without simultaneously considering how this cognitive ability is imparted in a leadership role. Cognitive ability is most strongly related to leader effectiveness when stress is low and when leaders are directive (Judge, Colbert et al., 2004).

There is a great deal of confusion and misinformation in the popular press about terms such as cognitive ability, intelligence, reasoning, IQ, and the like. All tests of cognitive ability are related, that is, correlated with each other to some greater or lesser degree. There are many tests of cognitive ability, each with strengths and limitations, and each with a somewhat unique interpretation and meaning.

Our approach is pragmatic. We define Capacity very specifically as skill in detecting patterns and trends, even in ambiguous, contradictory, or otherwise “noisy” environments, and we use a test that is independent of language or tacit knowledge of a domain in order to provide global comparability of scores. Specifically, we use Korn Ferry’s Logical assessment, a well-established, effectively language-free cognitive ability test. Language-free cognitive assessments are important to global clients who wish to apply the same test anywhere in the world. Our Logical assessment is a non-verbal measure of Gf, fluid intelligence (Horn, 1980). Gf is the capacity to think logically and solve problems in novel situations, independent of language or tacit knowledge of a domain.

Any strong measure of fluid intelligence like our Logical assessment is likely to have a quirk to consider. It will be negatively correlated with age. This is because, when younger, we rely more on Gf for performance in novel learning situations and for adaptive behavior. As we get older, we rely more on crystalized intelligence, Gc (experience, knowledge, and deductive reasoning). People tend to lose Gf reasoning skills and increasingly rely on heuristics, tacit knowledge, and experience for adaptive problem solving over time. Because the Leadership Potential Report is intended for early career assessment of potential to advance multiple levels in leadership, we do not recommend assessing Capacity for cohorts whose current level is at or above Business Unit Leader. More details on our Logical assessment can be found in the Korn Ferry Cognitive Ability Assessments Research Guide and Technical Manual (in press).

Role Preferences

Not all individuals are interested in becoming managers and leaders. While some employees value advancement above all else, others value primarily the intrinsic excitement of work. In a survey of engineers conducted in the early 1980s (Guterl, 1984), approximately one-third of the respondents indicated a preference for a management career, one-third for a non-management career, and one-third were unsure. This study reported that many engineers preferred to continue doing engineering and to avoid taking on administrative responsibilities. In another survey of approximately 1,500 individuals employed in R&D, less than one-third of the survey participants preferred the managerial career over alternative career paths (Allen & Katz, 1986). These earlier survey findings are consistent with a more recent survey which found that only about one-third (34%) of workers aspire to leadership positions (CareerBuilder, 2014).

Schein (1987) concluded that individuals hold a wide variety of career interests, which he labeled career anchors, or orientations. These career orientations influence career choices, employer changes, other career-related decisions, and ultimately how the future of career is viewed. At a high level, these differences describe the contradiction between specialists and generalists in organizations (Gouldner, 1957; Kornhauser, 1962; Schein, 1987). Specialists are characterized by strong professional identification, craftsman-like view of skill mastery, and a singular area of expertise. Conversely, generalists are identified by a drive to ascend the managerial hierarchy and breadth of functional-area expertise.



Given these differences, organizations were recommended to subdivide their high potential pools and provide different approaches to manage and engage specialists and generalists (Cesare & Thornton, 1993; Dries & Pepermans, 2008).

We systematically conducted research to obtain a more thorough understanding of the motivations, interests, and values of different types of talented employees in organizations. We were especially interested in understanding the difference between high potential leaders and high potential professional employees. Our research effort consisted of two phases of investigation—one qualitative and one quantitative.

The qualitative research started with interviewing 31 high-performing, specialized employees recruited from several organizations. They were nominated by HR professionals of the participants’ respective companies. Participants represented a wealth of different industries and professions, including engineers, scientists, physicians, financial analysts, musicians, and clinical psychologists. Respondents were asked a series of questions relating to their role preferences, events that shaped their career decisions, things they like and don’t like at work, challenging experiences and the lessons learned, most memorable achievements, and future aspirations.

All interviews were taped and transcribed. The transcripts were then analyzed for emerging themes. An analysis of the findings provides a clear, descriptive picture of high professionals: most tend to have (and desire) a linear career path; they are intrinsically motivated by their job; they have a strong desire to be recognized and respected for their expertise. They cite administrative or “busy work” and office politics as the biggest disruptors to their effectiveness and express a desire for autonomy and independent decision making in their roles.

Based on information from the first phase of research, Korn Ferry turned to statistically measuring differences in role preference, motivation, and engagement for high potential leaders and high potential professionals. This second phase of research involved developing 30 forced-choice items for pilot testing. The instrument was designed to capture the dichotomous relationship between having an orientation toward either a generalist role (i.e., a role with leadership/management responsibility) or a specialist role (i.e., a role that requires functional expertise). The results were compelling. Three factors that define a continuum of role preference emerged from analysis on data collected from a global sample of 10,823 participants:

• Factor I: How people perceive the bounds of their role responsibilities. Some people, preferring roles in narrowly defined areas, commit to their profession and enjoy opportunities that best use their expertise. Others, believing they can be versatile and take roles in different areas, explore and try different careers. They enjoy doing things they haven’t done before.

• Factor II: Different approaches to success. Individuals on one end of the continuum want to develop expertise and pursue best outcomes. They go deep and enjoy working toward greater precision and accuracy. Individuals on the other end of the continuum prefer to leverage others’ expertise and strive for practical outcomes. They make timely decisions and like to get a lot of work done, even if the work is imperfect.

• Factor III: How people engage in work. Some people want to concentrate on a few priorities for high productivity, believing it most effective to get one task done before starting another. Others enjoy multitasking, preferring to get involved in many different activities and shifting easily from priority to priority.

Twelve items were retained to assess the three factors. The sum of the three factors creates a scale that can distinguish the career orientation between high potential leaders and high potential professional employees.



Clear response patterns emerged in our global study. Data from over 10,000 assessments show that the specialist orientation declines steadily along the organizational hierarchy. Individual contributors (ICs) have the highest specialist orientations. Top-level executives have the lowest specialist orientations. And this pattern emerged consistently in different international regions.



Section 4 Measurement approach

For the Leadership Potential Report, Korn Ferry uses a combination of measurement approaches. In assessing leadership traits and drivers, we deploy state-of-the-art measurement technology: Forced-Choice Item Response Theory (Brown, 2016; Brown & Maydeu-Olivares, 2011). Forced-Choice Item Response Theory (FC-IRT) is also leveraged in reporting Learning Agility and Derailment Risks, which are profiles or configurations of scores on traits or on traits and drivers. In measuring Experience and Role Preferences, we use traditional Classical Test Theory approaches.

In this section of the manual, we first introduce FC-IRT, explaining the challenges it attempts to address and describing what it is. Then, we briefly summarize the administration of the FC-IRT assessments that participants complete as part of Korn Ferry’s Potential solution. Timing and response formats are outlined. Next, the algorithms involved in scoring FC-IRT are presented. Finally, we provide an overview of how results are presented in the Leadership Potential Report.

Forced-Choice Item Response Theory

Forced-Choice Item Response Theory (FC-IRT) offers a number of advantages over more traditional approaches to measurement, such as Likert-type response scales or Forced-Choice measures grounded in Classical Test Theory (CTT).

In brief, the use of FC-IRT (Brown, 2016; Brown & Maydeu-Olivares, 2011) provides a methodology with the following advantages:

• Removes response styles and scale anchor issues by eliminating the Likert rating scale.

• Thwarts faking by eliminating the transparency of the Likert rating scale.

• Frees interpretation from specific scale content by estimating true trait level.

• Provides superior true score estimates of error.

• Takes item content into consideration when estimating true score, thus improving fidelity of the score to the trait construct.

Many participants report the forced-choice response format to be more engaging and challenging than typical single-item rating assessments—some even describe it as difficult. This is intentional. Most understand that this is a characteristic of an assessment designed to be more incisive. It takes unlimited control of responses away from the participant and requires hard choices. It is this feature that inhibits faking and improves the quality of the data.

Response distortion and faking

One of the challenges in the measurement of the characteristics of potential by self-assessment is controlling intentional response distortion—faking. To the extent possible, it is very important to capture data that reflects a participant’s true standing on characteristics related to advancement. Psychometricians have become increasingly concerned with the response distortions associated with self-reports using the Likert-style response formats commonly used in psychological measurement (Stark, Chernyshenko, Chan, Lee, & Drasgow, 2001). As use of self-assessments of personality has increased, respondents’ desire to present themselves in a positive light to gain advantage has become of greater concern. The growing availability of self-coaching materials and use of unproctored internet-based tests has further contributed to the potential for faking to be increasingly problematic (Sliter & Christiansen, 2012). Past approaches to handling response distortion have not been highly satisfactory.



Social desirability and/or “faking scales” have long been deployed to detect and address faking. However, this approach suffers from limitations. When faking is indicated, it is difficult to know how to proceed. In research settings, a completed assessment may simply be thrown out. In applied settings, coaches and decision makers may simply be warned that the results are perhaps untrustworthy and to proceed with caution. Yet others have attempted to use results from faking detection or social desirability scales to adjust observed scores in diverse ways (Goffin & Christiansen, 2003). Such methods, however, have been repeatedly criticized as being arbitrary and difficult to validate (McCrae & Costa, 1983; Goffin & Christiansen, 2003). Sacket (2011), summarizing the observations contained in an edited book on faking in CTT, concludes, “In sum, we are not yet at the point at which methods of identifying individuals engaging in faking are available for use in operational settings,” and “So, without a reliable and valid detection method, scores cannot be adjusted to remove the effects of faking.”

Forced-choice response formats have also been developed and employed to combat faking. These formats force respondents to endorse or choose one item over another. When scored traditionally, these formats make extreme high or extreme low endorsement of every item impossible. In addition to combating faking (Sackett & Lievens, 2008), forced-choice measurement can markedly reduce response bias, “halo” or leniency effects, and response variance attributable to individual response styles not immediately associated with item content (Bartram, 2007; Cheung & Chan, 2002). However, when used in combination with classical test theory scoring methods, traditional forced choice response formats always produce scale scores which are known as ipsative, that are problematically auto-correlated and interdependent (Brown & Maydeu-Olivares, 2011). That is, an individual’s score on one scale is, in very large part, a direct and artificial reflection of the person’s scores on the other scales included in the measure (Heggestad, Morrison, Reeve, & McCloy, 2006).

This dependency makes normative comparisons across individuals improper and violates the assumptions of many common psychometric statistics (Blinkhorn, Johnson, & Wood, 1988; Hammond & Barrett, 1996; Hough & Ones, 2001; Meade, 2004).

Forced-choice IRT models

Researchers in psychological measurement have sought to tackle the many problems associated with forced-choice measures. Stark, Chernyshenko, and Drasgow (2005) developed a pairwise preference ideal point model that addresses most related problems by pairing and presenting items with similar levels of social desirability and by employing scoring and parameter estimation methods that are shown to perform well under certain conditions vis-à-vis eliminating ipsative auto-correlation. Stark and Chernyshenko (2007) point out that the number of pairwise preference ratings needed to obtain reasonable person-score standard errors using this approach may be particularly high in non-adaptive testing situations. Hence, the Stark et al. (2005) pairwise preference model works and is markedly more efficient with computer assisted adaptive testing administration, wherein item presentation is customized according to real-time response patterns. Where fixed form administration is optimal or necessary, test administration and reliability using the Stark et al. (2005) model may require many more items than desirable, limiting its feasibility in applied organizational settings. Additional limitations associated with this model’s reliance on an ideal point measurement framework include the relative difficulty of writing items, the lack of invariance in parameter estimates and model fit when reversing item coding, and the apparent reduced accuracy of item parameter estimation (Brown & Maydeu-Olivares, 2011; Maydeu-Olivares, Hernandez, & McDonald, 2006).

As an alternative, Brown and Maydeu-Olivares (2011) developed a structured multidimensional forced-choice IRT model that addresses problems associated with faking, response bias, and ipsativity while also addressing some of the limitations of the paired preference Stark et al. (2005) model. The authors describe a model that is linear in differences between latent traits.



Briefly, and in very straightforward terms, we can now simultaneously overcome the limitations and maintain the benefits of forced-choice measurement by employing recent advances in Item Response Theory (IRT) modeling that takes advantage of the item format and produces scale scores that are truly normative. This model is the FC-IRT model (see Brown & Maydeu-Olivares, 2011).

The method uses large samples of people and statistical models to identify 1) item loadings, which indicate the degree to which each item is related to the underlying trait it is intended to measure, and 2) item difficulty, which relates to the likelihood of a person with a given level of a trait ranking the item as more like them vs. other items. These two pieces of information are used, along with the rankings a person provides, to generate scores. A more detailed and statistical explication of the methodology can be found in subsequent paragraphs and sections.

The latent traits are manifest by binary comparisons of items that are presented in the forced-choice blocks. The model rearranges forced-choice responses into a series of exhaustive binary comparisons, thereby allowing for components of non-ipsative trait measures to drive parameter estimation, scoring, and interpretation of person-scores. The model is novel in that it creates a relative independence among otherwise predictably auto-correlated forced-choice based construct scores. It is flexible in terms of forced-choice block sizes and is feasible in that parameters and scores can be estimated using existing popular statistical software packages, including Mplus (Muthen & Muthen, 2010). We also have developed a related R package (Zes, 2015; Zes, Lewis, & Landis, 2014) that similarly estimates the Brown and Maydeu-Olivares (2011) model and related extensions of it.

FC-IRT uses a two-stage approach. If a person prefers or ranks item 1 higher than item 2, then that person’s preference for item 1 is higher than item 2. For instance, in a block of four items, there are six possible comparisons to make between the items. After a series of blocks are administered, we can use the information provided by these comparisons to fit an IRT model. We first use the IRT model to obtain information about how difficult the items are to endorse (called item difficulty), and how well the items relate to the traits being measured (called item loadings). Then, using item difficulty and item loading data, we can accurately estimate a person’s normative scale score for each of the underlying traits measured by the assessment. By abstracting the ranking information into two separate yet linked statistical models, this two-stage approach allows us to take forced-choice item responses and produce normative scale scores free of the ipsative limitations of traditional forced-choice measurement.

Using FC-IRT, we can take advantage of the bias and faking resistance of the forced-choice item response format while eliminating the psychometric limitations of the classical test theory scoring methods.

This state-of-the-art technology allows true normative comparison of individuals while effectively controlling error resulting from idiosyncratic use of a response scale or from intentional faking (Brown & Maydeu-Olivares, 2011; Drasgow, Chernyshenko, & Stark, 2010).

Administration and timing

To receive a Leadership Potential Report, participants complete assessments online. Paper and pencil versions are not available. The assessment experience is extremely efficient, taking approximately 35 minutes if the participant’s Experience career walk-through is included. There are a number of branching items, and several items require multiple responses. The total number of responses will be approximately 240–280 for most participants. Additional assessment time is needed if Capacity is assessed. The assessments are available in many major business languages.

A variety of item formats are used, making the assessments more engaging to complete. The majority are FC-IRT item administration, which require choices to be made in a ranking format. It is more demanding than traditional rating formats. Many participants welcome this level of structure and the engagement required, correctly inferring that more accurate insights are possible.



FC-IRT item administration

The scales scored with FC-IRT are administered in forced-choice response format. Each construct type is grouped together in its own test form. Construct scores are estimated using a modification (Zes, 2015; Zes et al., 2014) of the Brown and Maydeu-Olivares (2011) FC-IRT model to arrive at construct estimates whose correlations are based on the nature of the constructs and not according to forced-choice item response format artifacts. Eight items were designed to tap each trait, and trait response blocks contain four items each. Each driver is measured using ten items, with items presented in response blocks of six items.

An example of a forced-choice multi-item block from the Drivers test form is shown in Table 3. This example illustrates that each response block is comprised of items measuring multiple scales within the domain. That is, for each trait or driver response block, there is no more than one item from each scale.

Table 3. Example six-item block

Item 1 Well-defined work objectives.

Item 2 Situations without a winner and a loser.

Item 3 Having high status within the organization.

Item 4 Avoiding meetings so I can focus on my work.

Item 5 Developing myself beyond work.

Item 6 Consistent direction in my career.

Upon seeing a block of items, participants are tasked with ranking the items from “Most” to “Least” on some continuum. Specifically, in this example, participants would be asked to rank the items from “Most preferred” to “Least preferred.”

Other item administration

Items capturing Experience and Role Preferences are administered in other formats. As mentioned previously, Experience information is captured with a brief career walk-through. Participants are asked to check what experiences they have had, and, for Key Challenges, what roles they have played. Role Preferences are measured with classic bi-polar pairs of items with descriptive anchors at each extreme end of the response scale. The response format is modern. Participants slide a target along the scale to indicate their degree of preference in each item pair.

FC-IRT scoring

To set the stage for the FC-IRT model, assume that we have a test composed of several six-item blocks (as with the Drivers test form, which has 10 six-item blocks) where each item in a given block measures a unique construct or dimension (much like the example in Table 3). Further, assume that participants are asked to rank the items from “Most preferred” to “Least preferred.” To model this setup using FC-IRT, we first employ a Thurstonian Comparative Model (Brown & Maydeu-Olivares, 2011; for the origin of this model, see Thurstone, 1927). Using this model, for a given block of six items there are six latent utilities/thresholds, ti. If a participant prefers or ranks item i larger than item j, then the utility for item i, ti is larger than the utility for item j, tj. This information can be coded in a comparative task as

Forced Choice Item Response Theory

To assess a candidate’s standing on scales from each of the three dimensions (Drivers, Competencies,

Traits), items are presented in multi-item blocks as shown below.

Item 1 Well defined work objectivesItem 2 Situations without a winner and a loser.Item 3 Having high status within the organization.Item 4 Avoiding meetings so I can focus on my work.Item 5 Developing myself beyond work.Item 6 Consistent direction in my career.

Figure 1: Block of Six Items

Upon seeing a block of items, candidates are tasked with ranking the items from “Most” to “Least” on some

continuum. Specifically, in this example, candidates would be asked to rank the items from “Most Preferred”

to “Least Preferred”1. Traditionally—that is, in Classical Test Theory (CTT)—information gathered in this

way induces a dependence among the items and scales known as ipsativity (Brown & Maydeu-Olivares, 2011).

One attractive feature of ipsative measurement is that response biases and “halo” effects are diminished

(Bartram, 2007; Cheung & Chan, 2002). However, due to the inherent item dependence, ipsative scales

are also known to produce problems for “score interpretation and for almost every conventional type of

psychometric analysis” (Brown & Maydeu-Olivares, 2011, p. 461; see also Baron, 1996). We can overcome

this limitation of ipsative measurement and still reap the benefits by employing a Forced Choice Item

Response Theory (FCIRT) method of modeling the items and scale relationships (Brown, 2010; Brown &

Maydeu-Olivares, 2011, 2012, 2013; Maydeu-Olivares & Brown, 2010).

To set the stage for the FCIRT model, assume that we have a test composed of several six item blocks

where each item in a given block measures a unique construct or dimension (much like the example in Figure

1). Further, assume that candidates are asked to rank the items from “Most Preferred” to “Least Preferred”.

To model this setup using FCIRT, we first employ a Thurstonian Comparative Model (Brown & Maydeu-

Olivares; for the origin of this model, see Thurstone, 1927). Using this model, for a give block of six items,

there are six latent utilities, ti. If a candidate prefers or ranks item i larger than item j, then the utility for

item i, ti is larger than the utility for item j, tj . This information can be coded in a comparative task as

yl =

1 if ti ≥ tj

0 if ti < tj

. (1)

1Note - for the Competency and Trait Dimensions, candidates are asked to rank blocks of items from “Most Like Me” to“Least Like Me”.

1



Using this coding, if the latent utility for item i is larger than the latent utility for item j, the observed response, yl is represented by 1 (i.e., yl = 1 denotes that item i is ranked higher than item j). Then, for a block of six items, there are fifteen possible comparative tasks. With this setup, we can model the comparative tasks a latent factor model. To do so, we first note that the observed response, yl, is dependent on a difference of two item utilities. This difference can be represented as a latent comparative response, yl* = ti - tj, such that

Using this coding, if the latent utility for item i is larger than the latent utility for item j, the observed

response, yl is represented by 1 (i.e., yl = 1 denotes that item i is ranked higher than item j). Then, for a

block of six items, there are 15 possible comparative tasks. With this setup, we can model the comparative

tasks a latent factor model. To do so, we first note that the observed response, yl, is dependent on a difference

of two item utilities. This difference can be represented as a latent comparative response, y∗l = ti − tj , such

that

yl =

1 if y∗l ≥ 0

0 if y∗l < 0. (2)

Because we are assuming that the items measure a latent construct we can model each item’s utility as a

linear function of the underlying latent construct as

ti = µi + λiηa + εi, (3)

where µidenotes the mean of the latent utility, λi denotes a factor loading, and ηa denotes a common

latent factor underlying the utility ti, and ei denotes a unique factor. Moreover, we assume that each

item measures one and only one latent trait, the common and unique latent constructs are orthogonal and

normally distributed, and that unique factors across items are orthogonal.

Notice from (??)–(??) that we have a nested latent structure. Specifically, we have modeled each observed

binary response as being dependent on a latent comparative response, which in turn is dependent on a

linear combination of an underlying latent trait. This nested latent structure is typically referred to as a

second-order factor model. As is well known (Takane & De Leeuw, 1987), many IRT models are equivalent

to factor models of dichotomous variables. As shown by Brown and Maydeu-Olivares (2012), through

reparameterization, we can recast the second-order factor model as a first-order Thurstonian IRT model.

To reparameterize the model, we rewrite each latent comparative response as

y∗l = (ui − uj) + λiηa − λjηb + ei − ej . (4)

If we assume for the moment that the two traits are known, or conditioned on, and recalling the assumption

that the traits and unique factors are normally distributed, then the item characteristic function for preferring

item i over item j for a person can be written as

P (yl = 1|ηa, ηb) = NCDF

(ui − uj) + λiηa − λjηb√

Ψ2i +Ψ2

j

, (5)

2

Because we are assuming that the items measure a latent construct, we can model each item’s utility as a linear function of the underlying latent construct as






that

yl =

1 if y∗l ≥ 0

0 if y∗l < 0. (2)





















Ψ2i +Ψ2

j

, (5)

2

where μi denotes the mean of the latent utility, λi denotes a factor loading/discrimination, ηa denotes a common latent factor underlying the utility ti, and εi denotes a unique factor. Moreover, we assume that each item measures one and only one latent trait, that the common and unique latent constructs are orthogonal and normally distributed, and that unique factors across items are orthogonal.

Notice from (1), (2), and (3) that we have a nested latent structure. Specifically, we have modeled each observed binary response as being dependent on a latent comparative response, which, in turn, is dependent on a linear combination of an underlying latent trait. This nested latent structure is typically referred to as a second-order factor model. As is well known (Takane & De Leeuw, 1987), many IRT models are equivalent to factor models of dichotomous variables. As shown by Brown and Maydeu-Olivares (2012), we can recast the second-order factor model as a first-order Thurstonian IRT model via reparameterization.







that

yl =

1 if y∗l ≥ 0

0 if y∗l < 0. (2)





















Ψ2i +Ψ2

j

, (5)

2

If we assume for the moment that the two traits are known, or conditioned on, and recalling the assumption that the traits and unique factors are normally distributed, then the item characteristic function for preferring item i over item j for a person can be written as






that

yl =

1 if y∗l ≥ 0

0 if y∗l < 0. (2)





















Ψ2i +Ψ2

j

, (5)

2



where ψi² is the variance of item i uniqueness. Notice that the function is a standard normal ogive, which in this case is an IRT model that is dependent on two latent traits. Using this setup, the observed ipsative measurement model is transformed, loses its ipsative characteristics, and becomes a normative latent IRT model.

Scale score reporting

With the exception of Experience, raw scores are normed against a target role leadership level and reported as sten scores on a scale such as the one in Figures 4 and 5. The two aspects of Experience—Perspective and Key Challenges—are normed against current level. Sten scores are a standardized representation of raw scores relative to a norm. In stens, the mean is 5.5 and the standard deviation is 2.0. The target level is aspirational, typically two or three levels above the current level. Relative to that target norm:

• Scores at 5 or above 5 are regarded as strengths.

• Scores below 5 are development opportunities.

• Scores at 2 or below are clear opportunities for growth.

Figure 4. Sample report scale: Traits, Learning Agility, Experience, Drivers

5 6 7 8 9 10

In the case of Derailment Risks, scores above 8 indicate a profile of scores that may indicate a risk for derailing behaviors.

Figure 5. Sample report scale: Derailment Risks

9 10

Target levels and norms

Clients should carefully choose the target level for which potential is being assessed. We recommend this aspirational level should be two to three levels up from the current level of those being assessed. Target levels are briefly defined in Table 4. The norm groups used for scoring (with the exception of Experience) are also shown. Complete norm demographic descriptions are found in Appendix B “Norm descriptions.” Current norms are global, with representation from all regions.

The goal in choosing a target level is to provide differentiation of talent which requires a relatively “high bar.” There is little value in specifying Mid-Level Leader on down as a target level. Individual Contributors are not leaders, so no Individual Contributor target level norm is available. Individual Contributor norms are solely used for scoring Experience for current Individual Contributors. For most leaders and experienced Professional/Individual Contributors, the Team Lead and First Level Leader target levels are probably not appropriate. However, where Entry Level Individual Contributors are the population, those norms may be appropriate. However, a more demanding, higher-level norm is recommended.



Table 4. Target level definitions

Target level Definition NormChief Executive Officer/Top Organizational Executive

Serves as the senior most leader of a business or organization. In the public sector, includes the Director, Secretary, or Administrator of an agency or department.

Top Business or Organizational Group Executive


Serves as the senior most leader of a large business line or group of business lines (e.g., service lines, product lines, divisions, regions), or as a C-suite executive (e.g., Chief Financial Officer, Chief Operating Officer). In the public sector, includes senior executives who lead large groups of service lines or products directly related to the mission, or top-tier senior executive roles (e.g., Deputy Director, Under Secretary, or Chief Operating Officer of an agency or division).


Senior/Top Functional Executive Serves as the senior most leader of a business support or mission support function (e.g., Finance, HR, Facilities).

Senior/Top Functional Executive

Business or Organizational Unit/Division Leader

Leads a line of business (e.g., service line, product line, division, region). In the public sector, includes senior executives who lead large service or product groups directly related to the agency mission.

Business or Organizational Unit/Division Leader

Functional Leader Leads a function (e.g., Accounting, Compensation & Benefits, Marketing, IT), as opposed to a line of business (e.g., service line, product line, region). In the public sector, includes senior executives who play key leadership roles within mission support groups (e.g., Finance, HR, Procurement, Security).

Functional Leader

Mid-Level Leader Supervises one or more first-line supervisors/managers.

Mid-Level Leader

First Level Leader Guides the direction of a group or team of Individual Contributors and serves as their formal manager/supervisor.

First Level Leader

Team Lead Guides the direction of a team of Individual Contributors, or serves as its key subject-matter expert, without having formal supervisory/managerial responsibilities for team members.

First Level Leader

Individual Contributor/Professional Supports a business or organization without supervising others as part of his/her role.

Individual Contributor/Professional (Experience only)



Section 5 Technical characteristics

Assessments must have strong technical characteristics to offer meaningful insight into talent management questions. In this section, we describe the reliability and validity of the assessments used in the Potential solution. An overview of our approach to normative comparisons is also provided. The results demonstrate the technical robustness of Korn Ferry’s state-of-the-art assessments.

Reliability

Reliability is the consistency or dependability of a score. It provides an estimate of the amount of error, or noise, in the test’s measurement. In principle, if a test was completely reliable, all of the variation in people’s scores would be due to their true scores—all signal. In practice, there is always some noise in test scores, similar to there being some static, interference, or background noise on a phone call. Thus, reliability can be thought of as the signal portion of a signal-to-noise ratio for a test.

There is no exact determination of reliability, all estimates are—estimates. There are a number of ways to estimate reliability, such as test-retest (multiple administrations), or various internal methods (single administrations). The primary technical requirement is for a quality estimate of the proportion of true score (non-error, non-noise) included in a measure. Test-retest is a relatively poor estimate of the proportion of true score in a measure. This is because 1) it is not clear what retest time period is correct for this purpose, 2) it is not possible to preclude true change on a measured characteristic during the interim period, and 3) it is not possible to remove the effects of any process learning, assessment memory, retest boredom, or other effects from the retest results. Crocker and Algina (1986) conclude, “In view of these issues, it is probably sensible to assume that the test-retest coefficients probably present a somewhat inaccurate estimate of the theoretical reliability coefficient.” We focus, then, on the superior single measurement event procedures.

In Classical Test Theory (CTT), a single measurement event indicator of reliability is used to characterize the reliability of a test, typically, Cronbach’s α (Cronbach, 1951). It has been demonstrated that Cronbach’s α tends to be a conservative, lower-bound estimate of reliability (Osburn, 2000), which is one of the reasons it is so commonly used. A sufficiently high estimate of Cronbach’s α implies other types of reliability estimates will be even higher. Typically, in CTT terms, a value of .70 or higher is considered good in personality testing (Nunnally & Bernstein, 1994).

In Item Response Theory (IRT), there is no direct analog to Cronbach’s α, however, there is a similarly accurate approach (Maydeu-Olivares & Brown, 2010). In IRT, an estimate of error in a test can be calculated for each and every possible score along the full range of scores. For a conservative estimate comparable to Cronbach’s α, we calculate the proportion of true score included in the measure in scores across the full range of scores. We have computed from this an “average” reliability for each trait scale. These averages were computed by estimating the IRT score and error variances. The IRT score variance was estimated by computing the variance of computed IRT scores; the error variance was estimated by averaging the squared Standard Error of Measurement across the trait range. With an estimate of the IRT score and error variance in hand, the reliability, r’

tt, is estimated as the ratio of true score variance (IRT score variance minus error variance) to total score variance (IRT score variance). Acceptable reliabilities were observed for each of the scales (r’

tt ≥ .65 in every case).

Reliability estimates are presented for each scale in Table 5. The FC-IRT scales are estimated as described above. Role Preferences is a CTT scale and is estimated as Cronbach’s α. Note that the Experience scales are not included. This is because these measures do not involve estimates of a latent trait, but directly reflect back in summary form the participant’s experiences reported by the participant in the career walk-through.



Table 5. Reliability estimates

Leadership Traits

Persistence .78

Tolerance of Ambiguity .75

Assertiveness .77

Optimism .71

Learning Agility

Mental Agility .83

People Agility .82

Change Agility .82

Results Agility .82

Situational Self-Awareness .65

Drivers

Collaboration .80

Power .77

Challenge .78

Derailment Risks

Volatile .78

Micro-Managing .79

Closed .81

Capacity

Problem Solving .82

Role Preferences .67

Note. The reliability estimate for Problem Solving (measured with Korn Ferry’s Logical assessment) is based on marginal reliability (Green, Bock, Humphreys, Linn, & Reckase, 1984).

Construct validity

Correlation of the scales of one instrument with similarly conceptualized and defined scales of another instrument is considered evidence of construct validity (AERA, 2014). We have a sample of N = 481 persons who have taken the FC-IRT scales and the Global Personality Inventory (GPI) scales (SHL, 2001). The GPI is a 37-scale assessment of adult personality. The GPI scales are not necessarily similarly conceived or defined as the FC-IRT scales and tend to be somewhat more narrowly defined than the FC-IRT scales included in the Leadership Potential Report. GPI scale definitions are found in Appendix C.

CTT and FC-IRT self-assessment methods are quite different in response format and scoring procedures. One would not necessarily expect high correlations between administrations of even the same scales and items using the two methods due to 1) item ranking vs. Likert rating data collection, 2) simple arithmetic scoring of CTT vs. two-stage linear model scoring in IRT, 3) practical caps on correlations due to imperfect reliability of scales, and 4) the fake resistance of FC-IRT methodology.

In our case, we would expect moderate correlations between conceptually related scales using different instruments, items, and scoring methodologies and a pattern of correlations which generally support the construct validity of Leadership Potential Report scales. The full correlation matrix is 37 x 18. Due to its size, it is presented in Appendix D, along with GPI scale definitions, found in Appendix C. The results, however, are recapped in Table 6 with notable positive and negative correlations from the full matrix of correlations. All values are statistically significant p < .05.



Table 6. Notable correlations between the Leadership Potential Report and GPI scales

Global Personality Inventory scale

Leadership Potential Report scale Most notable scales with positive r Most notable scales with negative r

Leadership Traits

Persistence Self-Awareness/Insight

Energy Level

Influence

.23

.23

.22


Adaptability

Desire for Achievement

Risk-Taking

Energy Level

Innovation

.45

.30

.27

.22

.25

Attention to Detail

Passive-Aggressive

Micro-Managing

-.28

-.26

-.25

Assertiveness Taking Charge

Influence

Sociability

Energy Level


.54

.36

.33

.23

.29

Optimism Stress Tolerance

Sociability

Optimism

Trust

.32

.20

.25

.21

Micro-Managing

Negative Affectivity

-.23

-.29

Learning Agility

Mental Agility Innovation

Adaptability


Thought Focus

Vision

Risk-Taking

.42

.45

.32

.28

.31

.30

People Agility Influence

Sociability

Empathy

Consideration

Social Astuteness

.36

.36

.36

.30

.28

Independence -.20

(table continued)





Change Agility Adaptability

Risk-Taking


Energy Level

Innovation

Stress Tolerance

.50

.39

.32

.30

.24

.21

Attention to Detail

Micro-Managing

Passive-Aggressive

-.21

-.23

-.29

Results Agility Energy Level


Taking Charge

.33

.32

.21

Negative Affectivity

Passive-Aggressive

-.22

-.21


Stress Tolerance .29

Drivers

Collaboration Interdependence

Sociability

.29

.24

Independence

Intimidating

Passive-Aggressive

Ego Centered

-.30

-.24

-.24

-.21

Power Desire for Advancement

Taking Charge

Influence

Ego Centered

.44

.26

.22

.21

Challenge Desire for Achievement

Energy Level

Risk-Taking

Taking Charge

Adaptability

Influence

Desire for Advancement

Sociability

.38

.32

.30

.27

.27

.25

.25

.23

Passive-Aggressive -.24

(table continued)

Table 6. Notable correlations between the Leadership Potential Report and GPI scales (continued)





Derailment Risks

Volatile Stress Tolerance

Vision

Social Astuteness

-.37

-.22

-.20

Micro-Managing Micro-Managing .16 Attention to Detail -.23

Closed Micro-Managing

Passive Aggressive

Attention to Detail

.30

.28

.22

Adaptability


Risk-Taking

Energy Level

-.43

-.23

-.22

-.21

Role Preferences

Adaptability

Energy Level

Risk-Taking


.31

.24

.22

.20

Attention to Detail -.21

Note. All correlations statistically significant p < .05, N = 481.

Table 6. Notable correlations between the Leadership Potential Report and GPI scales (continued)



Criterion-related validity

Relationship with advancement

Criterion-related validity is evidence for the relationship of a measure to desired outcomes. Table 7 displays effect sizes for the differences in score for Individual Contributors against each higher organizational level. This is the foundation of differentiation of those who have advanced from those who have not.

An effect size is a quantitative measure of the strength of a phenomenon, in this case, the standardized mean difference between groups, Cohen’s δ. An effect size can be interpreted as small, medium, or large depending on its context. A commonly used interpretation is as follows: an effect size of 0.2 is considered a small effect, 0.5 a medium effect, and 0.8 and up a large effect (Cohen, 1988).

As shown in Table 7, bold indicates a statistically significant difference. The results show that there were progressively higher average scores for persons who have advanced to the next level for most scales, and, as expected, progressively lower scores for Derailment Risks scales. Most measures have a medium to large effect size.

Table 7. Effect sizes for different position levels

(Individual Contributor N > 551)

Individual Contributor/Professional to First Level Leader(N = 3212)

Individual Contributor/Professional to Mid-Level Leader(N = 4915)

Individual Contributor/Professional to Functional Leader(N = 4657)

Individual Contributor/Professional to Business or Organizational Unit/Division Leader (N = 4526)

Individual Contributor/Professional to Senior/Top Functional Executive(N = 3131)

Individual Contributor/Professional to Top Business or Organizational Group Executive(N = 1438)

Persistence 0.05 0.19 0.28 0.35 0.41 0.51


0.21 0.37 0.50 0.59 0.66 0.82

Assertiveness 0.21 0.33 0.37 0.54 0.57 0.71

Optimism 0.04 0.14 0.22 0.30 0.33 0.38

Mental Agility 0.10 0.25 0.39 0.47 0.53 0.68

People Agility 0.11 0.30 0.33 0.46 0.45 0.51

Change Agility 0.15 0.28 0.40 0.59 0.60 0.73

Results Agility 0.15 0.30 0.39 0.46 0.48 0.57


0.10 0.15 0.23 0.28 0.27 0.30

Collaboration 0.16 0.14 0.19 0.25 0.20 0.29

Power 0.02 0.02 0.00 0.12 0.11 0.15

Challenge 0.06 0.13 0.13 0.35 0.29 0.45

Perspective 0.04 0.15 0.39 0.38 0.55 0.69

Key Challenges -0.26 0.02 0.24 0.54 0.57 0.96

Volatile -0.14 -0.24 -0.31 -0.37 -0.38 -0.43

Micro-Managing -0.12 -0.22 -0.27 -0.23 -0.30 -0.31

Closed 0.15 0.08 -0.02 -0.10 -0.15 -0.24

Role Preferences 0.06 0.12 0.17 0.32 0.35 0.46



The progression of score changes by level can be seen clearly in percentile terms in Figure 6. Individual Contributor is the baseline and is represented at the 50th percentile.

Figure 6. Progressive increase (decrease) in scores with level

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Persis

tenc

e

Toler

ance

of A

mbig

uity

Asser

tiven

ess

Optimism

Menta

l Agilit

y

People

Agility

Chang

e Agilit

y

Resul

ts A

gility

Situat

iona

l Self

-Awar

enes

s

Collabora

tion

Power

Challe

nge

Persp

ectiv

e

Key C

halle

nges

Volatile

Micro-M

anag

ing

Close

d

Role Pre

fere

nces

IC FLL MLL BULFunctional Sr Functional Sr Executive

As previously mentioned, Korn Ferry’s Logical assessment was chosen as the measure of Capacity because it is a well-established effectively language-free test, a feature important to global clients. As is true of all tests of cognitive ability, this assessment carries with it an elevated possibility of adverse impact against traditionally disadvantaged ethnic groups. It also has a modest negative correlation with age at -.18 (see Table 8), suggesting Logical scores go down with age.

Table 8. Correlation between Logical scores and age

Age

Logical scores -.18

Note. Correlation statistically significant p < .01, N = 69176.

Reaching a minimum threshold for raw reasoning ability relative to other leaders is considered desirable for leadership success, and the Logical test can provide information in that area.



Prediction of Work Engagement

Work Engagement, the amount of discretionary effort a respondent is willing to expend toward their work, is a critical component of leadership performance. In a recent meta-analysis, Work Engagement has been demonstrated to provide substantial prediction of task and contextual work performance (Mr = .43 and .34, respectively), as well as prediction of organization commitment (Mr = .31) (Christian, Garza, & Slaughter, 2011).

In Table 9, correlations of Leadership Potential Report scales and Work Engagement are displayed. In addition to the association with advancement described above, many of the scales included provide economically valuable prediction of Work Engagement. This indicates that, in addition to differentiating by organizational level, many scales also differentiate those who are likely to be better, committed performers and more likely to expend discretionary effort. For Work Engagement, correlations are reported both raw and corrected for the reliability of the criterion variable (Spearman, 1904). All correlations are statistically significant.

Multiple regression using Work Engagement as the dependent variable and the 18 scales in Table 9 as the predictors indicates the Leadership Potential Report variables account for a meaningful part of the variance in Work Engagement, Multiple R = .436, (F [18, 26589] = 356.587, p <.000).

Table 9. Correlations of scales with Work Engagement (N = 54048)

Work Engagement,r’tt = .72raw corrected

Persistence .24 .28


.22 .26

Assertiveness .19 .22

Optimism .19 .22

Mental Agility .25 .26

People Agility .15 .18

Change Agility .23 .27

Results Agility .37 .42


.08 .09

Collaboration .14 .16

Power .05 .07

Challenge .24 .28

Perspective .11 .12

Key Challenges .17 .20

Volatile -.12 -.14

Micro-Managing -.06 -.07

Closed -.19 -.22

Role Preferences

-.05 -.06

Note. All correlations statistically significant p < .05.



Appendix A. Frequently asked questions

1. What is the research behind Item Response Theory (IRT)?

It is a complex technology. In Classical Test Theory (CTT), for self-report personality assessment, it is typical to use Likert ratings (responding on an item or question on a 1-to-5 or 1-to-7 ordered scale, e.g., Strongly agree to Strongly disagree) despite a number of limitations:

• Response styles (e.g., acquiescence bias, social desirability bias, preferring the center of the scale vs. the extremes of the scale).

• Scale anchors may be interpreted differently by respondents.

• Intentional faking in an effort to “game” or manipulate impressions cannot be controlled.

• Interpretation of scores is dependent on a particular norm and on the specific scale content.

• The estimated reliability and true score estimate error are the same for all participants.

• Item difficulty is not taken into account in scoring.

Tests based on IRT can overcome these issues by:

• Removing response styles and scale anchor issues by eliminating the rating scale.

• Thwarting faking by eliminating the transparency of the Likert rating scale.

• Freeing interpretation from specific scale content by estimating true trait level.

• Providing true score estimates of error at the individual level.

• Taking item content into consideration when estimating true score, thus improving fidelity of the score to the trait construct.

Methodologies developed at the University of Barcelona by Brown and Maydeu-Olivares (2011) have opened up an IRT methodology for forced-choice items formats using a Thurstonian paired-comparison measurement model. This model presents items in multi-item blocks (two or more items) and asks respondents to rank the items. Traditionally, such response formats in CTT produced results that were ipsative (items dependent on one another) with clearly detrimental effects. The state-of-the-art Forced-Choice Item Response Theory (FC-IRT) overcomes these issues.

The FC-IRT method of scoring uses a two-stage approach. It begins with applying a mathematical model for preferences. If a person prefers item 1, or ranks it higher than item 2, then that person’s preference for item 1 is higher than item 2. For a given block of four items, there are six possible comparisons to make between the items. After a series of blocks are administered, we can use the information provided by these comparisons to fit an IRT model. We first use the IRT model to obtain information about how difficult the items are to endorse (called item difficulty), and how well the items relate to the traits being measured (called item loadings). Then, using these two pieces of information, we can accurately estimate a person’s true score for each of the underlying traits measured by the assessment. By abstracting the ranking information into two separate yet linked statistical models, this two-stage approach allows us to take forced-choice item responses and produce normative scale scores free of the ipsative limitations of traditional forced-choice measurement. Using FC-IRT, we can take advantage of the bias and faking resistance of the forced-choice item response format while eliminating the psychometric limitations of the classical test theory scoring methods.



Korn Ferry scientists completed extensive simulations and pilots to:

• Test the underlying assumptions of the model and methods.

• Ensure that the new model did indeed overcome the prior psychometric insufficiencies of forced-choice methods.

• Develop extensions of the model to test the effects of intra- and inter-block “item cross talk” to better understand when item parameters may not be robust.

• Ensure that error estimates are meaningful.

The results from these statistical simulations and from live pilot data affirmed that the FC-IRT methodology is a superior technology in the ways described above.

2. Why do we select levels based on current vs. target? How do each of these show up in reporting? Whose responsibility is it to select each categorization?

The Leadership Potential Report is an assessment of the capacity and interest to develop the qualities required for effective performance in significantly more challenging leadership roles. Inherent in that is the assumption that this potential will play out over many years, or even decades.

A person’s Current Level is simply where they are situated today. It is used only to provide norms for Experience scales. This is because it would be unreasonable to expect persons at a First Level Leader level to have experiences normally gained further along in a career. Current level is set by the client for each group of participants at the time assessments are set up.

Target Level provides a common reference norm for clients to use to compare participants. It is important to select a target level that is two or three levels above the person’s current level. This sets a high bar and contemplates the possibility—even the likelihood—of that person advancing two or three or more levels beyond current level over the course of several years.

3. Do the Key Challenges remain the same in the instrument regardless of the target level selected by the client?

Yes, the Key Challenges remain the same. It is the amount and depth of participation in such challenges that vary with the norms for level.

4. What are the key differences between the Korn Ferry Assessment of Leadership Potential (KFALP) and the Leadership Potential Report?

• The seven signposts, while still conceptually relevant, are being removed from reporting. We found client focus on the signposts parts of the report distracted from the truly actionable and more meaningful data contained in each scale.

• There are several differences between these offerings. First, the KFALP measured five Leadership Traits: Focus, Persistence, Tolerance of Ambiguity, Assertiveness, and Optimism. The Leadership Potential Report includes all of these traits except Focus. Focus was discontinued due to a lower association with advancement than anticipated and some confusion about its interpretation.

• Awareness and Situational Self-Awareness were both reported in KFALP, whereas in the Leadership Potential Report, Awareness was eliminated, and Situational Self-Awareness is reported as part of the Learning Agility construct. Self-Awareness proved to have too little relationship to more manifest measures of self-awareness, such as self-other agreement in 360 feedback.

• Key components of the KFALP Advancement Drive measure of the motivation to advance have been unpacked and reported as individual Drivers scales (Collaboration, Power, and Challenge) to improve the nuance of reporting.



• Korn Ferry’s Logical assessment is used to measure Capacity in the Potential solution, rather than Raven’s which was used in KFALP.

• The Core Experience scale from the Experience section of KFLAP is not reported in the Leadership Potential Report; it was too closely tied to simple advancement. The Perspective and Key Challenges Experience scales remain.

• Larger global norms data are now available for each level.

5. Why are sten scores used rather than percentiles in the Leadership Potential Report?

• The sten distribution is normal; it is shaped like a normal curve. Scores are shown from 1-10, based on the sten distribution. We report these stens in single digits. The midpoint of the sten distribution is 5.5. Most people have scores between 4-7. Stens keep intact the normal distribution of raw scores.

• On the other hand, percentile distributions are rectangular; they have a definite disadvantage, in that scores toward the middle in a percentile-based report are exaggerated, and at the extremes minimized. Differences are often over-interpreted. For instance, the range of a percentile at 20th and 80th would appear quite extreme. In fact, each of the scores in raw terms is well within the middle third of the range of possible scores and closer to an average score than a very low or very high score.

• Stens and percentiles are not comparable.

6. Is a group report available?

A group report is not available at this time.

7. I’ve heard that most of the Talent Acquisition and Talent Management solutions available through the Korn Ferry Assessment Solution use Success Profiles; why doesn’t the Potential solution?

• Success Profiles are associated with specific jobs and fit with their requirements. The Leadership Potential Report focuses on a target level. Therefore, a norm group with a broader audience is used as the benchmark rather than a Success Profile.

• Remember our definition of potential: “The capacity and interest to develop the qualities required for effective performance in significantly more challenging leadership roles” and the intended uses of the report.

8. What scales are likely to stay constant over time for an individual? What are the biggest areas to see the most change?

None are constant, as all can change over time, but at different rates.

• Experience is the most directly malleable through proactively seeking broader experiences.

• Drivers can change with career and life stage as leaders decide to pursue more or less challenge or try a new path and discover a new passion.

• The Learning Agility, Leadership Traits, and Derailment Risks scales are trait-based. Traits change within an individual as they mature, but perhaps more slowly than Experience or Drivers. Traits have a big influence on behavior, as does work situation, learned self-regulation skills, fatigue, and other state factors. It is important to remember that traits are not fixed and do not directly determine behavior. Development is a matter of changing habits over time. With time and effort, change in habits is reflected in traits.



9. What, if any, differences in norms have we seen in regard to different regions, countries, cultures? Do results look similar across different regions, countries, or cultures?

We do not currently offer region-based norms for the Leadership Potential Report. We may offer geography-based norms at some point in the future as sufficient sample sizes by level are available. We are keenly interested, though, in understanding any differences across cultures.

In general, based on analyses of available data:

• Differences appear small, that is, there are far greater differences between persons within culture than between averages for different cultures.

• Relative to the sten reporting scale, it is very unlikely that an individual’s score would change more than +/- 1 point on the reporting scale based on a region comparison vs. a comparison with our global norm as reported.

10. Why are scores on the KFALP and the Leadership Potential Report different for the same scales?

• The reporting scale is different, moving away from percentiles in KFALP to stens in the Leadership Potential Report. They are not comparable.

• In the Leadership Potential Report, norms have been updated for all scales, which will result in somewhat different scores even if the reporting scale is constant.

• In addition, Results Agility and Mental Agility have been upgraded to broaden construct coverage and improve reliability.

• Derailment Risks scoring has been updated to be a better reflection of the construct—in other words, the score profiles that suggest each derailing characteristic.



Appendix B. Norm descriptions

General norm descriptions for the Leadership Potential Report.

First Level Leader N = 4912

Mean age: 39.2, SD = 7.2

Gender: Males 70.5%, Females 29.5%

Highest level of education completedSome Secondary Education 0.9%

Secondary/High School Graduate 4.4%

Trade/Technical Education 2.2%

Associate’s Degree/Diploma 5.8%

Undergraduate Degree (Bachelor’s, etc.) 47.4%

Postgraduate Degree (Master’s, etc.) 32.8%

Doctorate/Professional (Ph.D., M.D., etc.) 4.9%

Primary industryAdvanced Technology 8.6%

Consumer Goods 7.2%

Distribution Services 1.0%

Education & Training 0.8%

Energy & Utilities 15.2%

Financial Services 8.8%

Government 1.2%

Healthcare & Biological Sciences 20.8%

Industrial & Manufacturing 18.9%

Media & Entertainment 0.5%

Nonprofit 0.1%

Professional & Business Services 5.4%

Real Estate & Property Management 1.0%

Research & Development 0.3%

Retail 2.5%

Telecommunications 4.9%

Travel, Hospitality & Leisure 0.7%

Multi-Industry Holding Companies 2.0%



Current functional areaAdministrative Services 1.3%

Communications 1.1%

Consulting 3.3%

Creative 0.5%

Customer Service 3.3%


Executive & General Management 0.9%

Finance & Control 7.0%


Healthcare 1.3%

Human Resources 5.4%

Information Technology 10.9%

Legal 1.8%

Manufacturing 4.5%

Marketing 3.9%

Operations 11.9%

Public Safety & Military 0.6%

Research, Development & Engineering 11.5%

Retail 0.9%

Sales 17.0%

Strategic Planning 2.9%

Multiple Functions 5.8%

Company typePublicly Traded 54.5%

Subsidiary of Publicly Traded 9.7%

Privately Held 19.5%

Government Organization-Agency 4.1%

Government Owned Enterprise 10.3%

Nonprofit 1.3%

Self-Employed 0.6%

Mean years in management: 6.1, SD = 5.5



Global regionAnglosphere 23.9%

APAC 11.7%

East Europe 3.8%

Germanic 3.4%

Gulf 12.9%

India 8.7%

Japan 18.5%

Latin America 3.5%

Nordic 0.7%

Romance 9.2%

Russia 2.2%

No Data 2.0 %

Ethnicity (US only, optional)Hispanic or Latino 6.6%

White 76.4%

Black or African American 4.6%

Native Hawaiian or Other Pacific Islander 0.2%

Asian 10.0%

American Indian or Alaska Native 0.7%

Two or more races (more than one of the above) 1.6%



Mid-Level Leader N = 4916

Mean age: 42.9, SD = 6.9










Consumer Goods 6.9%





Government 0.6%




Nonprofit 0.1%




Retail 3.1%







Communications 1.0%

Consulting 4.4%

Creative 0.3%






Healthcare 1.2%



Legal 1.9%

Manufacturing 6.0%

Marketing 4.4%

Operations 10.4%



Retail 1.2%

Sales 13.2%








Nonprofit 1.6%

Self-Employed 0.3%





APAC 20.0%

East Europe 1.7%

Germanic 3.3%

Gulf 9.3%

India 11.9%

Japan 8.4%

Latin America 3.0%

Nordic 0.8%

Romance 7.7%

Russia 2.7%

No Data 2.3 %


White 72.5%



Asian 16.8%





Functional Leader N = 4658

Mean age: 43.8, SD = 7.0










Consumer Goods 7.5%





Government 0.8%




Nonprofit 0.2%




Retail 2.3%







Communications 1.5%

Consulting 3.4%

Creative 0.3%






Healthcare 1.2%



Legal 2.5%

Manufacturing 4.5%

Marketing 5.9%

Operations 9.8%



Retail 0.5%

Sales 7.5%








Nonprofit 1.8%

Self-Employed 0.2%





APAC 20.1%

East Europe 1.7%

Germanic 5.2%

Gulf 6.1%

India 8.4%

Japan 3.6%

Latin America 5.5%

Nordic 1.4%

Romance 8.8%

Russia 2.0%

No Data 3.0 %


White 78.4%



Asian 10.5%





Business or Organizational Unit/Division Leader N = 4526

Mean age: 45.1, SD = 6.9










Consumer Goods 7.0%





Government 1.0%




Nonprofit 0.3%




Retail 3.4%







Communications 0.6%

Consulting 2.7%

Creative 0.2%






Healthcare 1.6%



Legal 1.1%

Manufacturing 5.2%

Marketing 4.0%

Operations 11.3%

Public Safety, Military 0.2%


Retail 1.5%

Sales 13.7%








Nonprofit 3.5%

Self-Employed 0.3%





APAC 14.3%

East Europe 2.1%

Germanic 4.3%

Gulf 7.9%

India 8.5%

Japan 2.0%

Latin America 8.8%

Nordic 1.6%

Romance 12.1%

Russia 2.7%

No Data 3.2 %


White 77.6%



Asian 12.5%





Senior/Top Functional Executive N = 3131

Mean age: 46.0, SD = 7.0










Consumer Goods 7.5%





Government 0.8%




Nonprofit 0.4%




Retail 3.5%







Communications 1.8%

Consulting 1.3%

Creative 0.2%






Healthcare 1.1%



Legal 4.2%

Manufacturing 2.6%

Marketing 3.9%

Operations 9.2%



Retail 0.7%

Sales 7.8%








Nonprofit 3.1%

Self-Employed 0.3%





APAC 11.1%

East Europe 1.9%

Germanic 6.3%

Gulf 5.9%

India 4.7%

Japan 0.4%

Latin America 13.4%

Nordic 1.9%

Romance 16.5%

Russia 4.4%

No Data 1.9 %


White 83.7%



Asian 6.3%





Top Business or Organizational Group Executive N = 1438

Mean age: 48.2, SD = 6.7










Consumer Goods 7.5%





Government 1.1%




Nonprofit 0.6%




Retail 4.7%







Communications 1.0%

Consulting 1.3%

Creative 0.1%






Healthcare 1.0%



Legal 2.6%

Manufacturing 2.0%

Marketing 2.9%

Operations 6.5%



Retail 0.9%

Sales 6.0%








Nonprofit 3.8%

Self-Employed 0.6%





APAC 9.5%

East Europe 2.9%

Germanic 5.1%

Gulf 4.2%

India 3.5%

Japan 1.9%

Latin America 10.3%

Nordic 2.9%

Romance 22.9%

Russia 6.2%

No Data 1.1 %


White 83.8%


Native Hawaiian or Other Pacific Islander No Data

Asian 5.4%





Appendix C. Global Personality Inventory (GPI) definitions

Thinking Factor

ThoughT AgiliTyThis is a measure of the tendency to be open both to multiple ideas and to using alternative modes of thinking.

innovATion This is a measure of the tendency to produce unique and original things.

ThoughT FocusThis is a measure of the tendency to understand ambiguous information by analyzing and detecting the systematic themes in the data.

vision This is a measure of the tendency to have foresight in one’s thinking.

Planning and Execution Factor

ATTenTion To DeTAil This is a measure of the tendency to be exacting and precise.

Work Focus This is a measure of the tendency to be self-disciplined in one’s approach to work.

Facilitating Leadership Factor

TAking chArge This is a measure of the tendency to take a leadership role.

inFluence This is a measure of the tendency to get others to view and do things in a certain way.

Derailing Leadership Factor

ego cenTereD This is a measure of the tendency to be self-centered and appear egotistical.

MAnipulATion This is a measure of the tendency to be self-serving and sly.

Micro-MAnAgingThis is a measure of the tendency to over-manage once a person has advanced to higher levels of management.

inTiMiDATing This is a measure of the tendency to use power in a threatening way.

pAssive-AggressiveThis is a measure of the tendency to avoid confronting others, conveying acceptance or cooperation and yet appearing to behave in uncooperative and self-serving ways.

Interpersonal Factor

sociAbiliTy This is a measure of the tendency to be highly engaged by any social situation.

consiDerATion This is a measure of the tendency to express care about others’ well-being.

eMpAThyThis is a measure of the tendency to understand what others are experiencing and to convey that understanding to them.

TrusT This is a measure of the tendency to believe that most people are good and well intentioned.

sociAl AsTuTenessThis is a measure of the tendency to accurately perceive and understand the meaning of social cues and use that information to accomplish a desired goal.

energy level This is a measure of the tendency to be highly active and energetic.

iniTiATive This is a measure of the tendency to take action in a proactive rather than reactive manner.

Desire For AchieveMenT This is a measure of the tendency to have a strong drive to realize personally meaningful goals.

(table continued)



Self-Management Factor

ADApTAbiliTy This is a measure of the tendency to be open to change and considerable variety.

openness This is a measure of the tendency to accept and respect the individual differences of people.

negATive AFFecTiviTyThis is a measure of the tendency to be generally unsatisfied with many things, including but not limited to work.

opTiMisM This is a measure of the tendency to believe that good things are possible.

eMoTionAl conTrol This is a measure of the tendency to be even-tempered.

sTress TolerAnceThis is a measure of the tendency to endure typically stressful situations without undue physical or emotional reaction.

selF-conFiDence This is a measure of the tendency to believe in one’s own abilities and skills.

iMpressing This is a measure of the tendency to try to make a good impression on others.

selF-AWAreness This is the tendency to be aware of one’s strengths and weaknesses.

inDepenDence This is a measure of the tendency to be autonomous.

coMpeTiTiveness This is a measure of the tendency to evaluate one’s own performance in comparison to others.

risk-TAking This is a measure of the tendency to take chances based on limited information.

Desire For ADvAnceMenTThis is a measure of the tendency to be ambitious in the advancement of one’s career or position in organizational hierarchy.

Collective Orientation Factor

inTerDepenDence This is a measure of the tendency to work well with others.

DuTiFulness This is a measure of the tendency to be filled with a sense of moral obligation.

responsibiliTy This is a measure of the tendency to be reliable and dependable.

Global Personality Inventory (GPI) definitions (continued)



Appendix D. GPI and Leadership Potential Report scale correlations

Pers

iste

nce

Tole

ranc

e of

Am

big

uity

Ass

erti

vene

ss

Op

tim

ism

Men

tal A

gili

ty

Peop

le A

gili

ty

Res

ults

Ag

ility

Cha

nge

Ag

ility

Situ

atio

nal S

elf -

Aw

aren

ess

Col

lab

orat

ion

Pow

er

Cha

lleng

e

Pers

pec

tive

Key

Cha

lleng

es

Vol

atile

Mic

ro-M

anag

ing

Clo

sed

Rol

e P

refe

renc

es

Adaptability .03 .45 .07 .19 .45 .06 .09 .50 .12 .09 -.03 .27 .16 .12 -.14 -.15 -.43 .31

Attention to Detail .16 -.28 -.02 -.09 -.26 -.05 .10 -.21 .01 -.05 .01 -.04 -.09 -.07 -.09 .23 .22 -.21

Social Astuteness .15 .11 .08 .09 .16 .28 .08 .13 .11 .05 .04 .14 .07 .07 -.20 -.06 -.11 .05

Competitiveness .07 .01 .17 .04 .05 -.02 .04 .03 .00 -.13 .14 .14 .09 .19 -.05 .08 -.01 -.02

Consideration -.02 .02 -.01 .07 .07 .30 .12 -.02 .08 .19 -.11 -.01 -.02 .02 -.11 -.14 -.07 -.02

Desire for Achievement .15 .30 .29 .08 .32 .05 .32 .32 .00 .03 .16 .38 .11 .20 -.05 -.02 -.23 .20

Desire for Advancement .15 .06 .22 .02 .07 .00 .12 .13 .00 -.06 .44 .25 .07 .08 -.05 .02 -.08 .12

Dutifulness .16 .03 .04 .13 .05 .03 .18 .02 .07 .07 -.06 .05 -.01 .09 -.10 .05 -.04 .02

Emotional Control .05 .05 .06 .08 .11 -.01 .03 .06 .07 -.12 .02 .09 .13 .17 -.16 .07 -.04 -.05

Ego Centered .03 -.12 .14 -.13 -.07 -.08 .03 -.07 -.08 -.21 .21 -.01 .03 .01 .06 .09 .19 -.11

Energy Level .23 .22 .23 .18 .21 .13 .33 .30 .07 .12 .09 .32 .11 .23 -.09 .03 -.21 .24

Empathy .04 .00 -.01 .06 .09 .36 .04 -.01 .12 .14 -.06 .02 .01 .04 -.15 -.15 -.04 -.04

Impressing .07 -.06 .06 .01 .01 .01 .03 -.05 .01 -.12 .05 .01 .04 .11 -.04 .06 .05 -.08

Independence -.08 -.10 -.07 -.14 -.08 -.20 .01 -.18 -.13 -.30 .04 -.10 .00 -.05 .12 .11 .20 -.09

Influence .22 .16 .36 .13 .24 .36 .19 .20 .12 .02 .22 .25 .04 .24 -.12 -.02 -.06 .12

Initiative .05 .02 .13 .04 .09 .00 .03 .05 .01 -.11 .08 .07 .07 .16 -.03 .05 -.04 -.02

Innovation .11 .25 .14 .06 .42 .16 .05 .24 .11 -.07 .03 .18 .11 .13 -.11 -.04 -.11 .08

Interdependence .09 .06 .07 .09 .11 .14 .02 .18 .08 .29 -.10 .05 .01 .10 -.09 -.12 -.18 .11

Intimidating .08 -.05 .12 -.03 -.04 -.05 -.08 -.02 -.03 -.24 .09 .04 .04 .11 .00 .09 .08 -.07

Manipulation -.09 -.09 .00 -.15 -.12 -.15 -.18 -.06 -.14 -.12 .14 -.02 -.11 -.11 .10 .05 .11 .02

Micro-Managing -.02 -.25 .06 -.23 -.23 -.12 -.03 -.23 -.10 -.19 .05 -.12 -.11 -.12 .13 .16 .30 -.14

Negative Affectivity -.14 -.13 -.08 -.29 -.11 -.14 -.22 -.12 -.11 -.19 .01 -.13 -.04 -.13 .17 .05 .16 -.16

Openness .06 .04 .11 .09 .11 .04 .06 .09 .03 -.06 .04 .09 .10 .17 -.08 .02 -.09 .00

Optimism .17 .21 .11 .25 .22 .19 .15 .20 .16 .10 .03 .15 .11 .14 -.17 -.08 -.18 .11

Passive-Aggressive -.10 -.26 -.19 -.19 -.27 -.18 -.21 -.29 -.12 -.24 .02 -.24 -.14 -.21 .13 .08 .28 -.19

Responsibility .06 -.01 .08 .05 .06 .00 .06 .01 .02 -.12 .05 .06 .09 .18 -.06 .04 -.01 -.04

Risk-Taking .17 .27 .19 .03 .30 .10 .09 .39 .05 -.08 .09 .30 .17 .13 -.04 -.04 -.22 .22

Self-Awareness .23 -.01 .12 .18 .01 .13 .13 .05 .17 .04 .05 .04 .12 .21 -.19 .02 -.03 .03

Self-Confidence .06 .01 .11 .05 .08 -.01 .04 .03 .02 -.13 .07 .07 .08 .17 -.06 .06 -.01 -.03

Sociability .12 .08 .33 .20 .12 .36 .14 .19 .16 .24 .15 .23 .13 .14 -.14 -.03 -.17 .14

Stress Tolerance .14 .19 .01 .32 .15 .05 .13 .21 .29 .04 .00 .12 .12 .13 -.37 .04 -.14 .11

Thought Agility .05 .00 .08 .03 .07 .01 .01 .01 .02 -.11 .03 .04 .07 .17 -.05 .04 -.02 -.05

Taking Charge .19 .16 .54 .09 .18 .14 .21 .20 .12 .03 .26 .27 .11 .28 -.16 .06 -.06 .18

Thought Focus .17 .15 .14 .05 .28 .09 .14 .15 .12 -.06 .02 .13 .07 .25 -.15 .01 -.09 .06

Trust .08 .10 .03 .21 .09 .16 .15 .06 .13 .15 -.05 .02 .06 .10 -.09 -.10 -.07 .11

Vision .18 .19 .14 .03 .31 .17 .10 .22 .16 -.05 .02 .13 .13 .23 -.22 -.03 -.09 .04

Work Focus .09 -.06 .08 .04 .01 -.02 .05 -.03 .05 -.12 .04 .04 .08 .15 -.09 .09 .04 -.09 Note. N = 481, values > +/- .08 are statistically significant at p < .05.



References

Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121(2), 219-245. doi: 10.1037/0033-2909.121.2.219

Allen, T. J., & Katz, R. (1986). The dual ladder: Motivational solution or managerial delusion? R&D Management, 16(2), 185-197.

Alvesson, M., & Sveningsson, S. (2003). Good visions, bad micro-management and ugly ambiguity: Contradictions of (non-)leadership in a knowledge-intensive organization. Organization Studies, 24(6), 961-988. doi: 10.1177/0170840603024006007

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, D.C.: AERA.

Amos, B., & Klimoski, R. J. (2014). Courage: Making teamwork work well. Group & Organization Management, 39(1), 110-128. doi: 10.1177/1059601113520407

Anderson, C., & Kilduff, G. J. (2009). Why do dominant personalities attain influence in face-to-face groups? The competence-signaling effects of trait dominance. Journal of Personality and Social Psychology, 96(2), 491-503. doi: 10.1037/a0014201

Barrick, M. R., Parks, L., & Mount, M. K. (2005). Self-monitoring as a moderator of the relationships between personality traits and performance. Personnel Psychology, 58(3), 745-767. doi: 10.1111/j.1744-6570.2005.00716.x

Barrick, M. R., Stewart, G. L., & Piotrowski, M. (2002). Personality and job performance: Test of the mediating effects of motivation among sales representatives. Journal of Applied Psychology, 87(1), 43-51.

Bartram, D. (2007). Increasing validity with forced-choice criterion measurement formats. International Journal of Selection and Assessment, 15(3), 263-272. doi: 10.1111/j.1468-2389.2007.00386.x

Baum, J. R., & Locke, E. A. (2004). The relationship of entrepreneurial traits, skill, and motivation to subsequent venture growth. Journal of Applied Psychology, 89(4), 587-598. doi: 10.1037/0021-9010.89.4.587

Baumeister, R. F., & Leary, J. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497-529.

Bazerman, M. H., & Neale, M. A. (1983). Heuristics in negotiation: Limitations to effective dispute resolution. In M. H. Bazerman & R. J. Lewicki (Eds.), Negotiating in organizations (pp. 51-67). Thousand Oaks, CA: Sage Publications.

Bedford, C. L. (2011). The role of learning agility in workplace performance and career advancement (Unpublished doctoral dissertation). University of Minnesota, Minneapolis, MN.

Blinkhorn, S., Johnson, C., & Wood, R. (1988). Spuriouser and spuriouser: The use of ipsative personality tests. Journal of Occupational Psychology, 61, 153-162.

Brown, A. (2016). Item response models for forced-choice questionnaires: A common framework. Psychometrika, 81(1): 135-60.

Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71(3), 460-502.



Brown, A., & Maydeu-Olivares, A. (2012). Fitting a Thurstonian IRT model to forced-choice data using Mplus. Behavior Research Methods, 44(4), 1135-1147.

Bunker, K. A., Kram, K. E., & Ting, S. (2002). The young and the clueless. Harvard Business Review, 80(12), 80-87.

Campbell, J. P. (2012). Behavior, performance, and effectiveness in the twenty-first century. In S. W. J. Kozlowski (Ed.), The Oxford handbook of organizational psychology (Vol. 1, pp. 159-194). Oxford, England: Oxford University Press.

Cardon, M. S., Wincent, J., Singh, J., & Drnovsek, M. (2009). The nature and experience of entrepreneurial passion. The Academy of Management Review, 34(3), 511-532. doi: 10.5465/AMR.2009.40633190

Cardon, M. S., Zietsma, C., Saparito, P., Matherne, B., & Davis, C. (2005). A tale of passion: New insights into entrepreneurship from a parenthood metaphor. Journal of Business Venturing, 20, 23-45.

CareerBuilder. (2014). Majority of workers don’t aspire to leadership roles. Retrieved from http://www.careerbuilder.com/share/aboutus/pressreleasesdetail.aspx?sd= 9%2f10%2f2014&id=pr841&ed=12%2f31%2f2014

Carver, C. S., & Scheier, M. F. (2014). Dispositional optimism. Trends in Cognitive Sciences, 18(6), 293-299.

Cesare, S. J., & Thornton, C. (1993). Human resource management and the specialist/generalist issue. Journal of Managerial Psychology, 8(3), 31-40.

Charan, R., Drotter, S., & Noel, J. (2011). The leadership pipeline: How to build the leadership powered company. San Francisco, CA: Jossey-Bass.

Cheung, M. W. L., & Chan, W. (2002). Reducing uniform response bias with ipsative measurement in multiple-group confirmatory factor analysis. Structural Equation Modeling, 9(1), 55-77. doi: 10.1207/S15328007SEM0901_4

Christian, M. S., Garza, A. S., & Slaughter, J. E. (2011). Work engagement: A quantitative review and test of its relations with task and contextual performance. Personnel Psychology, 64(1), 89-136.

Church, A. H., & Rotolo, C. T. (2013). How are top companies assessing their high potentials and senior executives? A talent management benchmark study. Consulting Psychology Journal: Practice and Research, 65(3), 199-223. doi: 10.1037/a0034381

Church, A. H., Rotolo, C. T., Ginther, N. M., & Levine, R. (2015). How are top companies designing and managing their high-potential programs? A follow-up talent management benchmark study. Consulting Psychology Journal: Practice and Research, 67(1), 17-47. doi: 10.1037/cpb0000030

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

Collins, C. J., Hanges, P. J., & Locke, E. A. (2004). The relationship of achievement motivation to entrepreneurial behavior: A meta-analysis. Human Performance, 17(1), 95-117.

Cone, J. (2013). Tolerating ambiguity in four simple steps. Interaction Associates. Retrieved from http://interactionassociates.com/insights/blog/tolerating-ambiguity-four-simple-steps#.Vj_I2ZV0zug

Connolly, J. J. (2001). Assessing the construct validity of a measure of learning agility (Unpublished doctoral dissertation). Florida International University, Miami, FL.



Corporate Leadership Council. (2005). Realizing the full potential of rising talent. Washington, DC: Corporate Executive Board.

Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory Manual. Odessa, FL: Psychological Assessment Resources Inc.

Crandell, S., Hazucha, J., & Orr, J. E. (2014). Precision talent intelligence: The definitive four dimensions of leadership and talent (Report). Los Angeles, CA: Korn Ferry Institute.

Crandell, S., Orr, J. E., & Urs, L. (2015). Formative experiences may be key for CEO readiness. Los Angeles, CA: Korn Ferry Institute.

Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York, NY: CBS College Publishing.

Cronbach, L. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.

Dai, G., De Meuse, K. P., & Tang, K. Y. (2013). The role of learning agility in executive career success: The results of two field studies. Journal of Managerial Issues, 25(2), 108-131.

Dai, G., Tang, K. Y., & Feil, J. (2014, July). Fast-rising talent: Highly learning agile people get promoted at double speed (Report). Los Angeles, CA: Korn Ferry Institute.

Delgado-García, J. B., de Quevedo-Puente, E., & Fuente-Sabate, J. M. (2010). The impact of ownership structure on corporate reputation: Evidence from Spain. Corporate Governance: An International Review, 18(6), 540-556.

Deluga, R. J. (1998). American presidential proactivity, charismatic leadership, and rated performance. The Leadership Quarterly, 9(3), 265-291.

De Meuse, K. (2018, April). A meta-analysis of the relationship between learning agility and leader success. Poster presented at the 33rd Annual Conference of the Society for Industrial and Organizational Psychology. Chicago, IL. Retrieved from http://my.siop.org/Meetings/ProgramSearch18?EventID=1306

De Meuse, K. P., Dai, G., & Hallenbeck, G. S. (2010). Learning agility: A construct whose time has come. Consulting Psychology Journal: Practice and Research 62(2), 119-30.

De Meuse, K. P., Dai, G., Swisher, V. V., Eichinger, R. W., & Lombardo, M. M. (2012). Leadership development: Exploring, clarifying, and expanding our understanding of learning agility. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 280-286.

De Meuse, K. P., Dai, G., & Wu, J. (2011). Leadership skills across organizational levels: A closer examination. The Psychologist-Manager Journal, 14, 120-139.

De Pater, I. E., Van Vianen, A. E. M., Bechtoldt, M. N., & Klehe, U. (2009). Employees’ challenging job experiences and supervisors’ evaluations of promotability. Personnel Psychology, 62(2), 297-325.

Depue, R. A., & Collins, P. F. (1999). Neurobiology of the structure of personality: Dopamine, facilitation of incentive motivation, and extraversion. Behavioral and Brain Sciences, 22, 491-569.

DeRue, D. S., & Wellman, N. (2009). Developing leaders via experience: The role of developmental challenge, learning orientation, and feedback availability. Journal of Applied Psychology, 94(4), 859.

D’Mello, S. (2015). Keys to success. Los Angeles, CA: Korn Ferry Institute.



Dowell, B. E. (2010). Managing leadership talent pools. In R. F. Silzer & B. E. Dowell (Eds.), Strategy-driven talent management: A leadership imperative. San Francisco, CA: Jossey Bass.

Dragoni, L., Oh, I., VanKatwyk, P., & Tesluk, P. E. (2011). Developing executive leaders: The relative contribution of cognitive ability, personality, and the accumulation of work experience in predicting strategic thinking competency. Personnel Psychology, 64(4), 829-864.

Dragoni, L., Tesluk, P. E., & Oh, I. (2009). Understanding managerial development: Integrating developmental assignments, learning orientation, and access to developmental opportunities in predicting managerial competencies. Academy of Management Journal, 52(4), 731-743.

Dragoni, L., Tesluk, P. E., Russell, J. E. A., & Oh, I. S. (2009). Understanding managerial development: Integrating developmental assignments, learning orientation, and access to developmental opportunities in predicting managerial competencies. Academy of Management Journal, 52, 731-743.

Drasgow, F., Chernyshenko, O. S., & Stark, S. (2010). 75 Years after Likert: Thurstone was right! Industrial and Organizational Psychology: Perspectives on Science and Practice, 3(4), 465-476.

Drescher, M., Korsgaard, M. A., Welpe, I. M., Picot, A., & Wigand, R. T. (2014). The dynamics of shared leadership: Building trust and enhancing performance. Journal of Applied Psychology, 99(5), 771-783.

Dries, N., & Pepermans, R. (2008). “Real” high-potential careers: An empirical study into the perspectives of organizations and high potentials. Personnel Review, 37(1), 85-108.

Dries, N., & Pepermans, R. (2012). How to identify leadership potential: Development and testing of a consensus model. Human Resource Management, 51(3), 361-385. doi: 10.1002/hrm.21473

Dries, N., Vantilborgh, T., & Pepermans, R. (2012). The role of learning agility and career variety in the identification and development of high potential employees. Personnel Review, 41(3), 340-359.

Driskell, J. E., & Salas, E. (1992). Collective behaviour and team performance. Human Factors, 34, 277-288.

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087-1101. doi: 10.1037/0022-3514.92.6.1087

Dudley, N. M., Orvis, K. A., Lebiecki, J. E., & Cortina, J. M. (2006). A meta-analytic investigation of conscientiousness in the prediction of job performance: Examining the intercorrelations and the incremental validity of narrow traits. Journal of Applied Psychology, 91(1), 40-57. doi: http://dx.doi.org/10.1037/0021-9010.91.1.40

Elfenbein, H., Foo, M. D., White, J., Tan, H., & Aik, V. (2007). Reading your counterpart: The benefit of emotion recognition accuracy for effectiveness in negotiation. Journal of Nonverbal Behavior, 31(4), 205-223.

Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49(8), 725-747. doi: 10.1037/0003-066X.49.8.725

Eskreis-Winkler, L., Shulman, E. P., Beal, S. A., & Duckworth, A. L. (2014). The grit effect: Predicting retention in the military, the workplace, school and marriage. Frontiers in Psychology, 5(36). doi: 10.3389/fpsyg.2014.00036

Feldman, G., Hayes, A., Kumar, S., Greeson, J., & Laurenceau, J. P. (2007). Mindfulness and emotion regulation: The development and initial validation of the Cognitive and Affective Mindfulness Scale-Revised (CAMS-R). Journal of Psychopathology and Behavioral Assessment, 29(3), 177-190.



Furnham, A., Trickey, G., & Hyde, G. (2012). Bright aspects to dark side traits: Dark side traits associated with work success. Personality and Individual Differences, 52(8), 908-913. doi: 10.1016/j.paid.2012.01.025

Gaddis, B. H., & Foster, J. L. (2015). Meta-analysis of dark side personality characteristics and critical work behaviors among leaders across the globe: Findings and implications for leadership development and executive coaching. Applied Psychology: An International Review, 64(1), 25-54. doi: 10.1111/apps.12017

Galasso, A., & Simcoe, T. S. (2011). CEO overconfidence and innovation. Management Science, 57(8). doi: 10.1287/mnsc.1110.1374

Galinsky, A. D., Maddux, W. W., Gilin, D., & White, J. B. (2008). Why it pays to get inside the head of your opponent: The differential effects of perspective taking and empathy in negotiations. Psychological Science, 19(4), 378-384.

Gerstner, C., Hazucha, J., & Davies, S. (2012, April). Motivators: What is important to leaders at different levels. In S. E. Davies (Chair), Understanding and supporting transitions up the leadership ladder. Symposium presented at the Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.

Goffin, R. D., & Christiansen, N. D. (2003). Correcting personality tests for faking: A review of popular personality tests and an initial survey of researchers. International Journal of Selection and Assessment, 11(4), 340-344.

Gouldner, A. W. (1957). Cosmopolitans and locals: Toward an analysis of latent social roles. Administrative Science Quarterly, 2, 281-306.

Gratton, L. (2010). The future of work. Business Strategy Review, 21(3), 16-23.

Gratton, L., & Erickson, T. (2007). Eight ways to build collaborative teams. Harvard Business Review. Retrieved from https://hbr.org/2007/11/eight-ways-to-build-collaborative-teams

Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21(4), 347-360.

Guay, W., Taylor, D., & Xiao, J. (2014). Adapt or perish: Evidence of CEO adaptability to industry shocks [Working paper]. University of Pennsylvania: The Wharton School. Retrieved from https://www.mccombs.utexas.edu/~/media/Files/MSB/Departments/Accounting/TARC/2014/dan_taylor_paper.pdf

Guterl, F. (1984). Spectrum/Harris poll: The Career. IEEE Spectrum, 21(6), 59-63.

Haigh, E. A. P., Moore, M. T., Kashdan, T. B., & Fresco, D. M. (2011). Examination of the factor structure and concurrent validity of the Langer Mindfulness/Mindlessness scale. Assessment, 18(1), 11-26. doi: 10.1177/1073191110386342

Hammond, S., & Barrett, P. (1996). The psychometric and practical implications of the use of ipsative, forced-choice format, Questionnaires. Proceedings of the British Psychological Society Occupational Psychology Conference, January, 135-144. Leicester: BPS Press.

Haran, U., Ritov, I., & Mellers, B. (2013). The role of actively open-minded thinking in information acquisition, accuracy, and calibration. Judgment and Decision Making, 8(3), 188-201.

Harrison, S. H., Sluss, D. M., & Ashforth, B. E. (2011). Curiosity adapted the cat: The role of trait curiosity in newcomer adaptation. Journal of Applied Psychology, 96(1), 211-220.

https://hbr.org/2007/11/eight-ways-to-build-collaborative-teams



Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choice assessments of personality for selection: Evaluating issues of normative assessment and faking resistance. Journal of Applied Psychology, 91(1), 9-24. doi: 10.1037/0021-9010.91.1.9

Hogan, R. (1983). A socioanalytic theory of personality. In M. M. Page (Ed.), 1982 Nebraska symposium on motivation (pp. 55-89). Lincoln, NE: University of Nebraska Press.

Hogan, R., & Hogan, J. (2001). Assessing leadership: A view from the dark side. International Journal of Selection and Assessment, 9(1/2), 40-51.

Hogan, R., & Warrenfeltz, W. (2003). Educating the modern manager. Academy of Management Learning and Education, 2, 74-84.

Horn, J. L. (1980). Concepts of intellect in relation to learning and adult development. Intelligence, 4, 285-317.

Hough, L. M., & Ones, D. S. (2001). The structure, measurement, validity, and use of personality variables in industrial, work, and organizational psychology. In N. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), Handbook of industrial, work, and organizational psychology: Volume 1 Personnel psychology (pp. 233-277). London, England: Sage Publications.

Hough, L. M., Ones, D. S., & Viswesvaran, C. (1998). Personality correlates of managerial performance constructs. In R. C. Page (Chair), Personality determinants of managerial potential: Performance, progression and ascendancy. Symposium conducted at the 13th Annual Convention of the Society for Industrial and Organizational Psychology, Dallas, TX.

Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment, 9(1-2), 152-194.

Houlfort, N., Philippe, F. L., Vallerand, R. J., & Menard, J. (2014). On passion and heavy work investment: Personal and organizational outcomes. Journal of Managerial Psychology, 29(1), 25-45.

House, R. J., Spangler, W. D., & Woycke, J. (1991). Personality and charisma in the U.S. presidency: A psychological theory of leader effectiveness. Administrative Science Quarterly, 36(3), 364-396.

Howe, M. J. A. (1999). Genius explained. New York, NY: Cambridge University Press.

Hughes, R. L., Ginnett, R. C., & Curphy, G. J. (2008). Leadership: Enhancing the lessons of experience (6th ed.). New York, NY: McGraw-Hill.

Johnson, N. (2007). Self report measures of mindfulness: A review of the literature (Master’s thesis, Pacific University). Retrieved from http://commons.pacificu.edu/cgi/viewcontent.cgi?article=1145&context=spp

Judge, T. A., Bono, J. E., Ilies, R., & Gerhardt, M. W. (2002). Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87, 765-780. doi: 10.1037//0021-9010.87.4.765

Judge, T. A., Colbert, A. E., & Ilies, R. (2004). Intelligence and leadership: A quantitative review and test of theoretical propositions. Journal of Applied Psychology, 89(3), 542-552. doi: 10.1037/0021-9010.89.3.542

Judge, T. A., & Piccolo, R. F. (2004). Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology, 89(5), 755-768.

Kaiser, R. B., LeBreton, J. M., & Hogan, J. (2015). The dark side of personality and extreme leader behavior. Applied Psychology: An International Review, 64(1), 55-92. doi: 10.1111/apps.12024



Kam, C., & Meyer, J. P. (2012). Do optimism and pessimism have different relationships with personality dimensions? A re-examination. Personality and Individual Differences, 52, 123-127.

Karaevli, A., & Hall, D. T. (2003). Growing leaders for turbulent times: Is succession planning up to the challenge? Organizational Dynamics, 32, 62-79.

King, J. E., & Figueredo, A. J. (1997). The five-factor model plus dominance in chimpanzee personality. Journal of Research in Personality, 31(2), 257-271.

Kirschkamp, A. (2007). A contingency-based view of chief executive officers’ early warning behavior: An empirical analysis of German medium-sized companies. Deutscher Universitäts-Verlag | Gabler (GWV) Fachverlage GmbH, Wiesbaden.

Korn Ferry. Cognitive ability assessments research guide and technical manual. (in press). Minneapolis, MN: Author.

Korn Ferry. ViaEdge® technical manual. (2013). Minneapolis, MN: Author.

Kornhauser, W. (1962). Scientists in industry: Conflict and accommodation. Berkeley, CA: University of California Press.

Kruglanski, A. W., & Freund, T. (1983). The freezing and unfreezing of lay inferences: Effects on impressional primacy, ethnic stereotyping, and numerical anchoring. Journal of Experimental Social Psychology, 19, 448-468.

Kruglanski, A. W., Shah, J. Y., Pierro, A., & Mannetti, L. (2002). When similarity breeds content: Need for closure and the allure of homogeneous and self-resembling groups. Journal of Personality and Social Psychology, 83, 648-662.

Kruglanski, A. W., & Webster, D. M. (1996). Motivated closing of the mind: “Seizing” and “freezing.” Psychological Review, 103(2), 263-283. doi: 10.1037/0033-295X.103.2.263

Kumar, S., Feldman, G., & Hayes, A. (2008). Changes in mindfulness and emotion regulation in an exposure-based cognitive therapy for depression. Cognitive Therapy and Research, 32(6), 734-744. doi: 10.1007/s10608-008-9190-1

Langer, E. J. (2009). Counterclockwise: Mindful health and the power of possibility. New York, NY: Ballantine Books.

Laxson, E. N. (2018). Within and between person effects of learning agility: A longitudinal examination of how learning agility impacts future career success (Unpublished doctoral dissertation). Colorado State University.

Leary, M., Diebels, K., Davisson, E., Jongman-Sereno, K., Isherwood, J., Raimi, K., Deffler, S., & Hoyle, R. (2017). Cognitive and interpersonal features of intellectual humility. Personality and Social Psychology Bulletin, 43(6), 1-21.

Lee, D. Y., & Tsang, E. W. K. (2001). The effects of entrepreneurial personality, background and network activities on venture growth. The Journal of Management Studies, 38(4), 583-602.

Lee, R. A. (2012). Accelerating the development and mitigating derailment of high potential through mindfulness training. The Industrial-Organizational Psychologist, 49(3), 23-34.

Lewis, J. L., & Ream, R. K. (2012). The criterion-related validity of the Decision Styles assessment. Los Angeles, CA: Korn Ferry International.



Liu, D., Chen, X., & Yao, X. (2011). From autonomy to creativity: A multilevel investigation of the mediating role of harmonious passion. Journal of Applied Psychology, 96(2), 294-309. doi: http://dx.doi.org/10.1037/a0021294

Livingstone, H. A., & Day, A. L. (2005). Comparing the construct and criterion-related validity of ability-based and mixed-model measures of emotional intelligence. Educational and Psychological Measurement, 65(5), 757-779.

Lombardo, M. M., & Eichinger, R. W. (1989). Preventing derailment: What to do before it’s too late. Greensboro, NC: Center for Creative Leadership.

Lombardo, M. M., & Eichinger, R. W. (2000). High potentials as high learners. Human Resource Management, 39(4), 321-329.

Mann, R. D. (1959). A review of the relationships between personality and performance in small groups. Psychological Bulletin, 56(4), 241-270. doi: 10.1037/h0044587

Marinova, S. V., Moon, H., & Kamdar, D. (2013). Getting ahead or getting along? The two-facet conceptualization of conscientiousness and leadership emergence. Organization Science, 24(4), 1257-1276. doi: 10.1287/orsc.1120.0781

Martin, J., & Schmidt, C. (2010). How to keep your top talent. Harvard Business Review, 88(5), 54-61.

Maydeu-Olivares, A., & Brown, A. (2010). Item response modeling of paired comparison and ranking data. Multivariate Behavioral Research, 45(6), 935-974. doi: 10.1080/00273171.2010.531231

Maydeu-Olivares, A., Hernandez, A., & McDonald, R. P. (2006). A multidimensional ideal point item response theory model for binary data. Multivariate Behavioral Research, 41(4), 445-472.

Mayseless, O., & Kruglanski, A. W. (1987). What makes you so sure? Effects of epistemic motivations on judgmental confidence. Organizational Behavior and Human Decision Processes, 39, 162-83.

McCall, M. W., Jr., Lombardo, M. M., & Morrison, A. M. (1988). The lessons of experience: How successful executives develop on the job. Lexington, MA: Lexington Books.

McCauley, C. D., Ruderman, M. N., Ohlott, P. J., & Morrow, J. E. (1994). Assessing the developmental components of managerial jobs. Journal of Applied Psychology, 79(4), 544.

McClelland, D. C. (1965). Toward a theory of motive acquisition. American Psychologist, 20, 321-333.

McClelland, D. C., & Boyatzis, R. (1982). Leadership motive pattern and long-term success in management. Journal of Applied Psychology, 67(6), 737-743.

McClelland, D. C., & Burnham, D. H. (1976). Power is the great motivator. Harvard Business Review, 54, 100-110.

McCrae, R. R., & Costa, P. T. (1983). Social desirability scales: More substance than style. Journal of Consulting and Clinical Psychology, 51(6), 882-888. doi: 10.1037/0022-006X.51.6.882

McCrae, R. R., & Costa, P. T. (1987). Validation of the 5-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52, 81-90.

Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77, 531-552.

Moutafi, J., Furnham, A., & Paltiel, L. (2005). Can personality factors predict intelligence? Personality and Individual Differences, 38(5), 1021-1033. doi: 10.1016/j.paid.2004.06.023

Muthen, L. K., & Muthen, B. O. (2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthen & Muthen.



Nadkarni, S., & Barr, P. S. (2008). Environmental context, managerial cognition, and strategic action: An integrated view. Strategic Management Journal, 29(13), 1395-1427. doi: http://dx.doi.org/10.1002/smj.717

Northouse, P. G. (1997). Leadership: Theory and practice. Thousand Oaks, CA: Sage Publications, Inc.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge, T. A. (2007). In support of personality assessment in organizational settings. Personnel Psychology, 60(4), 995-1027. doi: 10.1111/j.1744-6570.2007.00099.x

Ones, D. S., Dilchert, S., Viswesvaran, C., & Salgado, J. F. (2010). Cognitive abilities. In J. L. Farr & N. T. Tippins (Eds.), Handbook of employee selection (pp. 255-275). New York, NY: Routledge/Taylor & Francis.

Osburn, H. G. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psychological Methods, 5(3), 343-355.

Ottati, V., Price, E., Wilson, C., & Sumaktoyo, N. (2015). When self-perceptions of expertise increase closed-minded cognition: The earned dogmatism effect. Journal of Experimental Social Psychology 61, 131-138.

Palaiou, K., Zarola, A., & Furnham, A. (2016). The dark side of personality predicts positive and negative work attitudes. Personality and Individual Differences, 88, 12-16. doi: 10.1016/j.paid.2015.08.029

Palanski, M. E., & Carroll, E. A. (2006). The role of behavioral integrity as an antecedent to leader emergence. Paper presented at the Academy of Management Annual Meeting, Atlanta, GA.

Papadakis, V., & Bourantas, D. (1998). The chief executive officer as corporate champion of technological innovation: An empirical investigation. Technology Analysis & Strategic Management, 10(1), 89-109.

Pepermans, R., Vloeberghs, D., & Perkisas, B. (2003). High potential identification policies: An empirical study among Belgian companies. Journal of Management Development, 22(8), 660-678.

Peterson, D. B., & Erdahl, P. (2007, April). Early identification and development of senior leadership talent: The secret insider’s guide. Workshop presented at the 22nd Annual Conference of the Society for Industrial and Organizational Psychology, New York, NY.

Philippe, F. L., Vallerand, R. J., Houlfort, N., Lavigne, G. L., & Donahue, E. G. (2010). Passion for an activity and quality of interpersonal relationships: The mediating role of emotions. Journal of Personality and Social Psychology, 98(6), 917-932. doi: 10.1037/a0018017

Price, E., Ottati, V., Wilson, C., & Kim, S. (2015). Open-minded cognition. Personality and Social Psychology Bulletin, 41(11), 1488-1504.

Quinones, M. A., Ford, J. K., & Teachout, M. S. (1995). The relationship between work experience and job performance: A conceptual and meta-analytic review. Personnel Psychology, 48(4), 887-910.

Ready, D., Conger, J., Hill, L., & Steckler, E. (2010). The anatomy of a high potential. Business Strategy Review, 21(3), 52-55.

Reeves, M., & Deimler, M. (2011). Adaptability: The new competitive advantage. Harvard Business Review Blog. Retrieved from https://hbr.org/2011/07/adaptability-the-new-competitive-advantage

Reio, T. G., Jr., & Wiswell, A. (2000). Field investigation of the relationship among adult curiosity, workplace learning, and job performance. Human Resource Development Quarterly, 11(1), 5-30.

http://dx.doi.org/10.1002/smj.717



Reynolds, D. H., McCauley, C. D., & Tsacoumis, S. (in press). A critical evaluation of the date of assessment and development for senior leaders. Industrial and Organizational Psychology: Perspectives on Science and Practice.

Rokeach, M. (1960). The open and closed mind. New York, NY: Basic Books.

Sackett, P., (2011). Faking in personality assessments: Where do we stand? In M. Ziegler, C. MacCann, & R. Roberts (Eds.), New perspectives on faking in personality assessments (pp. 330-344). Oxford University Press.

Sackett, P. R., & Lievens, F. (2008). Personnel selection. Annual Review of Psychology, 59, 419-450.

Schein, E. H. (1987). Individuals and careers. In J. W. Lorsch (Ed.), Handbook of organizational behavior (pp. 155-171). Englewood Cliffs, NJ: Prentice-Hall.

Segerstrom, S. C. (2007). Optimism and resources: Effects on each other and on health over 10 years. Journal of Research in Personality, 41(4), 772-786.

Shapira, Z. (1995). Risk taking: A managerial perspective. New York, NY: Russell Sage Foundation.

Sharpe, J. P., Martin, N. R., & Roth, K. A. (2011). Optimism and the Big Five factors of personality: Beyond Neuroticism and Extraversion. Personality and Individual Differences, 51, 946-951.

Sherrill, W. W. (2001). Tolerance of ambiguity among MD/MBA students: Implications for management potential. Journal of Continuing Education in the Health Professions, 21, 117-122.

SHL. (2001). Global personality inventory. Arlington, VA.

Silzer, R., & Borman, W. C. (2017). The potential for leadership. In D. G. Collings, K. Mellahi, & W. F. Cascio (Eds), Oxford Handbook of Talent Management. Oxford, UK: Oxford University Press.

Silzer, R., & Church, A. H. (2009). The pearls and perils of identifying potential. Industrial and Organizational Psychology, 2(4), 377-412.

Silzer, R., & Church, A. H. (2010). Identifying and assessing high potential talent: Current organizational practices. In R. F. Silzer & B. E. Dowell (Eds.), Strategy-driven talent management: A leadership imperative (pp. 213-280). San Francisco, CA: Jossey-Bass.

Silzer, R., Church, A., Rotolo, C., & Scott, J. (2016). I-O practice in action: Solving the leadership potential identification challenge in organizations. Industrial and Organizational Psychology, 9(4), 814-830. doi: 10.1017/iop.2016.75

Simons, T. L., & McLean Parks, J. (2000, August). The sequential impact of behavioral integrity on trust, commitment, discretionary service behavior, customer satisfaction, and profitability. Paper presented at the Academy of Management Annual Meeting, Toronto, Ontario, Canada.

Simonton, D. K. (1994). Greatness: Who makes history and why. New York, NY: Guilford Press.

Slan-Jerusalim, R., & Hausdorf, P. A. (2007). Managers’ justice perceptions of high potential identification practices. Journal of Management Development, 26(10), 933-950.

Sliter, K. A., & Christiansen, N. D. (2012). Effects of targeted self-coaching on applicant distortion of personality measures. Journal of Personnel Psychology, 11(4), 169-175.

Spain, S. M., Harms, P., & Wood, D. (n.d.). Stress, well-being, and the dark side of leadership [Unpublished manuscript]. Retrieved from https://binghamton.academia.edu/SethSpain



Spangler, W. D. (1992). Validity of questionnaire and TAT measures of need for achievement: Two meta-analyses. Psychological Bulletin, 112(1), 140-154.

Spearman, C. (1904) The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72-101.

Stark, S., & Chernyshenko. O. S. (2007). Adaptive testing with the multi-unidimensional pairwise preference model. In D. J. Weiss (Ed.), Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing. Retrieved from www.psych.umn.edu/psylabs/CATCentral

Stark, S., Chernyshenko, O. S., Chan, K. Y., Lee, W. C., & Drasgow, F. (2001). Effects of the testing situation on item responding: Cause for concern. Journal of Applied Psychology, 86(5), 943.

Stark, S., Chernyshenko, O. S., & Drasgow, F. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise-preference model. Applied Psychological Measurement, 29(3), 184-203.

Stogdill, R. M. (1948). Personal factors associated with leadership: A survey of the literature. Journal of Psychology, (25), 35-71.

Strosaker, G. (2010). Developing a tolerance for ambiguity. Constant Cogitation. Retrieved from http://gregstrosaker.com/2010/01/developing-a-tolerance-for-ambiguity/

Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52(3), 393-408.

Tang, K. Y., & Dai, G. (2013). The Leadership Architect® 2013 global norms report II: Career stallers and stoppers norms and analysis. Los Angeles, CA: Korn Ferry.

Tang, K. Y., Dai, G., & De Meuse, K. P. (2013). Assessing leadership derailment factors in 360° feedback: Differences across position levels and self-other agreement. Leadership & Organization Development Journal, 34(4), 326-343.

Tannenbaum, S. I. (1997). Enhancing continuous learning: Diagnostic findings from multiple companies. Human Resource Management, 36(4), 437-452.

Teodorescu, A., Furnham, A., & MacRae, I. (2017). Trait correlates of success at work. International Journal of Selection and Assessment, 25(1), 36-42.

Terman, L. M., & Oden, M. H. (1947). The gifted child grows up: Twenty-five years’ follow-up of a superior group (Vol. 4). Stanford University Press.

Tesluk, P. E., & Jacobs, R. R. (1998). Toward an integrated model of work experience. Personnel Psychology, 51, 321-355.

Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 79, 281-299.

Venus, M., Mao, C., Lanaj, K., & Johnson, R. E. (2012). Collectivistic leadership requires a collective identity. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5(4), 432-436.

Weick, K. E., & Roberts, K. H. (1993). Collective mind in organizations: Heedful interrelating on flight decks. Administrative Science Quarterly, 38(3), 357-381.

White, R. D. (2010). The micromanagement disease: Symptoms, diagnosis, and cure. Public Personnel Management, 39(1), 71-76. doi: 10.1177/009102601003900105



Wiggins, J. S. (Ed.). (1996). The five-factor model of personality: Theoretical perspectives. New York, NY: Guilford.

Wilson, E. (2015). Empathy is still lacking in the leaders who need it most. Harvard Business Review. Retrieved from https://hbr.org/2015/09/empathy-is-still-lacking-in-the-leaders-who-need-it-most

Winter, D. G. (1987). Leader appeal, leader performance, and the motive profiles of leaders and followers: A study of American presidents and elections. Journal of Personality and Social Psychology, 52(1), 196-202.

Yammarino, F. J., Salas, E., Serban, A., Shirreffs, K., & Shuffler, M. L. (2012). Collectivistic leadership approaches: Putting the “we” in leadership science and practice. Industrial and Organizational Psychology: Perspective on Science and Practice, 5(4), 382-402.

Yukl, G., & Mashud, R. (2010). Why flexible and adaptive leadership is essential. Consulting Psychology Journal: Practice and Research, 62(2), 81-93.

Zaccaro, S. J. (2001). The nature of executive leadership: A conceptual and empirical analysis of success. Washington, DC: American Psychological Association.

Zaccaro, S. J., White, L., Kilcullen, R., Parker, C., Williams, D., & O’Connor-Boes, J. (1997). Cognitive and temperament predictors of Army civilian leadership. Bethesda, MD: Management Research Institute.

Zes, D. (2015). KFmlfcirt: FC-IRT Multi-loadings. R package version 0.6.7.

Zes, D., Lewis, J., & Landis, D. (2014). kcirt: k-Cube Thurstonian IRT Models [Computer software]. https://cran.r-project.org/web/packages/kcirt/index.html



About Korn FerryKorn Ferry is a global organizational consulting firm. We help clients synchronize strategy and talent to drive superior performance. We work with organizations to design their structures, roles, and responsibilities. We help them hire the right people to bring their strategy to life. And we advise them on how to reward, develop, and motivate their people.

Talent Management Assessment Solution: Leadership Potential Report Research Guide and Technical Manual

Item Number KFTM-01 Version 18.1a—12/2018

Talent Management Assessment Solution · Talent Management Assessment Solution: Leadership...

Documents

Transcript of Talent Management Assessment Solution · Talent Management Assessment Solution: Leadership...