Effective integration of 360 degree feedback into the ... _360s_in_coaching.pdf · Effective...
Transcript of Effective integration of 360 degree feedback into the ... _360s_in_coaching.pdf · Effective...
The Coaching Psychologist, Vol. 4, No. 1, April 2008 7
© The British Psychological Society – ISSN: 1748–1104
Effective integration of 360 degreefeedback into the coaching process
Almuth McDowall & Rainer Kurz
Introduction to feedback processes
ESSENTIALLY, the term ‘Feedback’
stems from communications theory and
refers to a process where a ‘sender’,
relays a ‘message’ which is some information,
to a ‘recipient’ (McDowall, 2008). This basic
process is common to all feedback activities
and is outlined in Figure 1 (overleaf).
Feedback processes can take various forms
in interpersonal processes, as it can be deliv-
ered either orally, written or in electronic form.
Feedback can also originate from the ‘self’, as
cognitive and affective processes provide us
with information about how any task that we
are involved in is progressing (for a full review
see Kluger & DeNisi, 1996). In fact, structured
‘self-assessment’ is at the heart of most multi-
rater feedback processes, which we discuss
below both in terms of the extant research
literature but also the practical implications. In
addition, feedback will also originate from any
given task itself, which may provide signals of
frustration, boredom, enjoyment and so on.
Kluger and DeNisi (1996) integrated
these aspects into a Feedback Intervention
Theory [FIT] which generated a number of
testable propositions that urged researchers
to consider a complex array of factors in
future research and practice. Feedback is also
a crucial element of self-regulatory theories
of human motivation such as control theory
(see Carver & Scheier, 1981) as it allows us to
evaluate what we should do or avoid in rela-
tion to our internal goals and standards. We
cannot assume, however, that feedback will
always have positive effects, as many examples
exist to the contrary. A case in point is the
dissatisfaction with contemporary appraisal
processes in organisations (e.g. Jones, 2007).
In a similar vain Thorndike (cited in Kluger
& DeNisi, 1996) noted as early as 1913 that
giving feedback by the means of grades to
students changed their focus from wanting to
learn to the mark obtained.
Contemporary research has followed on
from this to corroborate that a focus on
performance (i.e. ‘marks’) as opposed to
mastery (i.e. learning) can be detrimental to
performance. This has clear implications for
practice as formative feedback, focused on
improvement and the future, is infinitely
This practical paper is based on a skills session as delivered at the British Psychological Society’s Special
Group in Coaching Psychology Conference 2007 held in London. The first part of our paper provides a
review of the extant research evidence on 360 degree feedback with focus on effects on individual
development, making explicit links to the implications for coaching practice throughout. We conclude that
360 degree feedback is primarily effective when conceptualised and utilised as a finely grained means of
instigating individual behaviour change and learning on job relevant attributes and facilitated by a skilled
feedback giver. This provides a clear rationale for its use in coaching. We outline how an actual profile can
be used as part of a coaching session, using the Saville Work Wave® Performance 360 as an example.
Keywords: 360 degree feedback, self-awareness, individual development, blind spots, proficiency,
feedback conditions, potential, aptitude, personality, styles, Saville Consulting Wave®.
A version of this paper was presented at the 3rd National Coaching Psychology Conference
in December, 2007.
8 The Coaching Psychologist, Vol. 4, No. 1, April 2008
Almuth McDowall & Rainer Kurz
more conducive than summative feedback
focused on past performance. We will return
to this aspect when discussing the implica-
tion for practice in the last section. Next, we
will outline the rationale and research
evidence for multi-source feedback
processes in detail.
360 degree feedback – its rationale andresearch evidenceA fairly comprehensive body of evidence on
feedback in an organisational context stems
from studies on multi-source-multi-rater
(MSMR) or 360 degree feedback (McDowall,
2008). This entails the planned comparison
of ratings from various sources, such as the
supervisor, subordinates, peers and also
occasionally internal and external
customers, on agreed work-based perform-
ance dimensions and interpersonal aspects.
Numerous 360 degree feedback tools can be
bought ‘off the shelf’ (akin to psychometric
tests albeit with greater variations in psycho-
metric quality). Examples are profiles that
are based on transformational models of
leadership such as the MLQ (Avolio et al.,
1995) or the TLQ (now ELQ, Alimo-
Metcalfe & Alban-Metcalfe, 2001, 2008).
At the other extreme of a continuum are
bespoke tools that are typically based on an
organisation’s competency framework. In
between are ‘half-way house’ tools that allow
limited customisation of existing items or
scales against a local framework. Reports are
almost always generated through a computer
system. They typically contain graphs, usually
bar charts, that show the ratings from various
parties on different dimensions and may
include structured narratives or free flowing
comments. There was a temporary backlash
against this emphasis on numerical informa-
tion some time ago as some people objected
to creating such a distinct and possibly
unhealthy emphasis on performance and
summative grading. A relatively recent study,
however, shows that managers in fact prefer
this format to narrative and potentially more
formatively orientated information (Atwater
& Brett, 2006).
The process of 360 degree feedback typi-
cally commences with a recipient self-assess-
ment on a set of items. Other raters are
Figure 1: The Feedback process.
Feedback message :
what the sender conveys to
the recipient ; usually related
to the effectiveness of the
recipient’s behaviour
Feedback Sender :
Coach, manager, but also the task and the
self
Feedback recipient :
The person who receives the feedback , in
coaching usually the coachee
What does the coach need to
understand about each of
these parties to implement
best feedback practice ?
nominated to complete the same items (or
sometimes a sub-set). The ‘other’ raters will
typically include the immediate superior or
boss, peers, subordinates and occasionally
also internal or external customers (see
Figure 2) or other stakeholders. Feedback
givers can be recruited in various ways. In
contexts where emphasis is put on formal
summative ratings, the organisation, or its
representatives tend to specify the other
raters. Where the focus is on formative devel-
opment, the focal individual will chose the
other raters.
When 360 degree feedback first
emerged, great hopes rested on this innova-
tive process, where beneficial outcomes as
depicted in Figure 2 were expected at
different levels. Multi-source feedback was
purported to enhance individual effective-
ness and learning, facilitate culture change
and increase effectiveness at a wider team
and organisational level which ultimately
would result in better services to customers.
The purpose of the actual feedback
process (which should be conducted by a
trained professional) is to assess and evaluate
how any gaps between different ratings may
have arisen and to explore them with the focal
individual. According to McDowall (2008)
effective 360 degree feedback should at the
very least meet the following conditions:
(a) It should provide a comprehensive and
valid measure of workplace behaviour.
(b)The learning through feedback from
different sources should prompt people
to change, and engage in relevant follow
up development activities.
(c) Feedback from each source will provide
valuable information in its own right.
We will now briefly review existing literature
from major occupational psychology jour-
nals to ascertain to what extent the above
conditions are met in practice focusing on
the individual level. We omit a wider discus-
sion of purported benefits at the team and
organisational level to focus on evidence
relevant to coaches.
Accuracy of ratings and 360 toolsMultiple raters should in theory provide
more accurate information than singular
ratings as a combination from various
The Coaching Psychologist, Vol. 4, No. 1, April 2008 9
Effective integration of 360 degree feedback into the coaching process
Figure 2: The 360 degree feedback process and purported benefits.
PeerPeer
Customer
SubordinateSubordinate
Boss
Self
Organisational benefits : productivity ,
effectiveness , culture change
Team benefits :
communication
understanding , empowerment
Customer benefits :
better service
Individual benefits :
greater self insight ,
effectiveness , development ,
potential realisation
sources should serve to cancel out any indi-
vidual bias. Fletcher and Baldry (1999) point
out that this may not necessarily be the case
especially if reward decisions such as promo-
tions, bonuses or pay-rises are contingent on
ratings. To assure quality Fletcher et al.
(1998) suggest that 360 measures should be
scrutinised and tested just like any other
psychometric tool to increase confidence
that the results will hold up across time or
across employees, and they in fact measure
what the organisation had intended to
measure. In our own practice, we have used
Fletcher et al.’s method to scrutinise existing
360 data sets, and have come to similar
conclusions in different contexts: (a) that
individual items often show poor discrimina-
tion (everybody gives similar ratings);
(b) that items may not be measuring the
competency that they have been assigned to;
and (c) that instruments generally are over-
long due to a number of redundant items.
Coaches need to bear this in mind when
deciding about which tools to use in their
own practice or when interpreting a pre-
existing profile. McDowall and Kurz (2007)
put forward that psychometric tools can be
judged effectively by their adherence to
psychometric principles (are instruments
reliable and consistent, do they measure
what they set out to measure, is an appro-
priate point of comparison available, are
they free from bias against certain groups).
This also applies to multi-source feedback
tools. We have summarised a 360 quality
check for coaches in Table 1.
360 and individual development360 degree feedback offers the unique
opportunity to compare ratings from
different sources against how people rate
themselves providing an indication of effec-
tiveness at work. The size and direction of
the gaps between self- and other ratings indi-
cate pressure points where dialogue could
facilitate development. The evidence
suggests that individuals who are self-aware,
in other words able to rate their respective
strengths and weaknesses in the same way
that other people rate them, perform opti-
mally in the workplace (e.g. Bass &
Yammarino, 1991; Yammarino & Atwater,
1993). Yammarino and Atwater (1997) put
forward four categories of agreement:
l In Agreement – Good (high ratings from
self and other).
l In Agreement – Poor (low ratings from
self and other).
l Under-rater (low ratings from self, high
ratings from others).
l Over-rater (high ratings from self, low
ratings from others).
Evidence suggests that Over-raters tend to
ignore criticism and discount failure (Bass &
Yammarino, 1991) and poor performance
outcomes are predicted for ‘In Agreement –
Poor’ raters who according the model have
not addressed any shortcomings. However,
self-awareness can be conceptualised as a
state (transient) rather than a trait. Evidence
shows that Over-raters demonstrate the
highest levels of improvement following 360
feedback (Atwater et al., 1995). This is
important to note for the use of 360 in
coaching. Very different questioning and
interview strategies may be required for an
Over-rater, who at first might be inclined to
disregard the information contained in the
feedback profile than for an Under-rater
who may lack self-confidence. Profiles that
include actual comments from raters are
helpful in this instance, as these provide
additional evidence that can feed into the
coaching process.
In all, the rationale for using 360 at the
individual level is to raise or corroborate
people’s level of self-awareness through feed-
back from different sources (McDowall,
2008). However, not all feedback informa-
tion is attended to equally. The credibility
and the rank of the feedback source matter,
as feedback from the boss generally has the
greatest impact (Bailey & Fletcher, 2002;
Gregura et al., 2003). Peer ratings appear to
vary considerably across time (Bailey &
Fletcher, 2002). Reports often focus more on
the pleasantness than the effectiveness of
their boss. Coaches need to bear these
10 The Coaching Psychologist, Vol. 4, No. 1, April 2008
Almuth McDowall & Rainer Kurz
The Coaching Psychologist, Vol. 4, No. 1, April 2008 11
Effective integration of 360 degree feedback into the coaching process
Table 1: Checking 360 degree feedback quality.
Level of Analysis Questions about the Tool Who provides the Information
The tool itself Is there any information available about: Tool publishers, or organisation
Reliability? (for bespoke tool).
Validity?
Comparison groups?
The job or role Which of the 360 dimensions are crucial This needs to be done through
to the focal individual’s job or role? job analysis, key consultation
or tools such as the job profiler.
The individual Is the information as presented in the Feedback discussion(s) between
profile correct? feedback giver (e.g. coach)
Which of the rater categories as explained and feedback recipient
above does the focal individual pertain to? (focal individual).
Do they over-, or under-rate?
How can gaps between self- and other
ratings be explained?
What has been learned from the
discussion of the profile?
How can this be taken forward?
The team Is there any evidence that use of The organisation,
360 degree feedback results in better e.g. commissioning managers.
communication, teamwork, etc.?
Are there any issues that need to be
targeted specifically?
Is this backed up by aggregation of
different profiles?
The organisation Is use of 360 linked to improved The organisation, test publisher
outcomes at the organisational level, or independent researchers
such as productivity, satisfaction, undertaking validation studies.
turnover, participation in training and
development?
Should any of the above provide focus
for the coaching process(es)?
multiple stakeholder perspectives in mind
when interpreting differences in ratings
from various sources. As boss’s ratings tend
to be weighed heavier by focal individuals,
this could be used to good advantage for
Over-raters, as criticism from this source is
more likely to be accepted and acted upon.
Will 360 degree feedback achieve lasting
impact? An early study (Hazucha et al.,
1993) found that self-awareness increased
following participation in 360 degree feed-
back, and that this in turn was related to
career progress. Engagement in follow-up
activities was contingent on support from
supervisors where those who felt supported
put more effort into their development and
engaged in more development activities.
The association between 360 degree
feedback ratings and follow-up develop-
mental activities tends to be small (Maurer et
al., 2002). A thorough review also found the
link between feedback ratings (both from
traditional appraisals and multiple sources)
and performance improvements to be negli-
gible (Smither et al., 2005). Effectiveness
improves, however, if the initial feedback
process is followed up and supported by
executive coaching, where notably better
performance evaluations are observed as
well as the setting and follow through of
concrete goals (Smither et al., 2003). As
reported in McDowall (2008) this also
applies outside the workplace in education
(Marsh & Roche, 1997). Taken together,
these results indicate that coaching is helpful
for initiating and embedding behaviour
change following the initial feedback
process. These findings can be explained in
the light of existing psychological theories of
motivation. Goal setting theory holds that
individuals are more motivated to initiate
and sustain behaviour if difficult but achiev-
able goals are set (for a full review see
Latham, 2006). Whilst goal setting theories
are primarily concerned with how cognitive
resources are energised and attention is
focused and sustained, control theory
(Carver & Scheier, 1981) takes a socio-cogni-
tive perspective. Individuals compare their
own internal goals against standards set by
others and strive to match these, which
could be feedback ratings in the case of 360
processes.
According to control theory, three condi-
tions have to be met for individuals to adjust
their behaviour. First, individuals must have
a goal or goals they are striving towards. In
the case of 360 degree feedback, these could
be general goals such as wanting to improve
the quality of working life, or specific goals
such as improving particular aspects of
competence (e.g. improving communication
in team work). Secondly, people must recog-
nise that certain aspects of their current
behaviour are not in line with these goals.
Lastly, they must be willing and able to adjust
their behaviour to meet these goals.
The basic tenets of control theory are
helpful when working with clients and 360
feedback profiles in coaching practice as
they can be communicated openly to a
coachee. As coachees may or may not agree
with the feedback received from others (and
the level of agreement), it is important to
convey that such information is essential for
the proactive regulation of our behaviour.
This is empowering for coaches, as control
theory presupposes human agency and thus
places the ‘ball back in the court of the
coachee’ for controlling behavioural
changes. It also fosters positive self-beliefs
which are crucial for 360 feedback effective-
ness (Maurer et al., 2002; Atwater & Brett,
2005). In addition, it is important that
people react positively to the feedback
process (Antonioni, 1996; Atwater & Brett,
2005) as a negative attitude makes people
reluctant to change.
For the above reasons, it is important that
360 processes are communicated well and
buy in is sought from everyone involved.
Mutual trust needs to be fostered. Without
trust, attitudes to the process are likely to be
defensive and negative, and may also result
in distorted ratings. To this extent, it is
important that the overall purpose of the
process is clearly agreed upfront. Whilst in
the US 360 ratings may feed into organisa-
tional decisions such as promotions or
rewards, in the UK use is largely limited to
developmental purposes, and many authors
would recommend this as best practice (see
Fletcher & Baldry, 1999). Where 360 degree
process is implemented with care, managers
value its thoroughness (Mabey, 2001), and
particularly appreciate the detailed graph-
ical and numerical information that is
usually detailed in the reports (Atwater &
Brett, 2006).
One issue that is generally problematic
both in 360 degree feedback but also in
appraisal is the question of who is actually
best placed to rate other people’s perform-
ance. For instance, line or senior managers
may have little idea of what an individual
actually does on a day-to-day basis. Thus, one
study found that behaviour change as meas-
ured by comparison of 360 degree ratings
over time was more closely related to initial
self-assessments than initial ratings from
other sources (Bailey & Austin, 2006). This
finding suggests that we should give as much
12 The Coaching Psychologist, Vol. 4, No. 1, April 2008
Almuth McDowall & Rainer Kurz
attention to self-evaluations as to ratings
from other sources, and that we should look
at underlying trait measures to explore the
reasons for performance issues.
Saville Consulting Wave®
Performance 360 As discussed above, the use of 360 tools in
coaching is particularly helpful where reli-
able evidence from the actual work context
is required. ‘Whilst managers who don’t
want to hear’ might brush off the discussion
of a traditional personality profile or intelli-
gence test yet evidence gathered from
various sources in the workplace is much
harder to ignore.
To measure underlying potential trait
measures can identify stable personality
characteristics through self report question-
naires and abilities through aptitude tests.
Traditional personality measures focus on
how people differ from each other on behav-
iour, behavioural preferences or characteris-
tics that effect their behaviours, but unlike
aptitude tests without indicating whether
there is a positive or negative impact on work
effectiveness of this difference. The lack of
validated links between traditional person-
ality dimensions and performance criteria
hence hampers an analysis of the gap
between potential and performance.
This is where robust instruments that are
based on cutting edge psychological
research are of particular merit and can
contribute valuable information to coach
and coachee alike as a sound 360 tool will
allow performance and potential to be
unraveled in fine granularity.
Saville Consulting tools have been built
specifically to predict and measure work
performance based on the design principle
of a ‘matched model’ where trait predictors
and work performance criteria are struc-
turally aligned spanning an assessment
continuum that ranges from dispositional
trait measures to work competency measures.
The Saville Consulting Wave® Perform-
ance 360 tool measures performance
through self and other ratings on inventories
featuring 36 Behaviour items, six Ability and
three Global Performance items that are
each rated on a seven-point effectiveness
scale. It derives from the initial Validation
Player instrument used by MacIver et al.
(2007) in the development of Saville
Consulting Wave to validate a pool of 214
facet scales against empirical criteria of job
performance and potential.
The relevance of the same behavioural,
ability and global performance areas can be
measured through the companion Impor-
tance 360 or Job Profiler multi-rater tools.
Potential for each area can be predicted
from related personality, aptitude and
competence attributes.
In Saville Consulting Wave® model
Performance 360 competency dimensions
and styles assessments are structurally
aligned and psychometrically linked through
Competency Potential equations that draw
on a broader range of valid facets to optimise
the empirical prediction through variables
outside the matched model. Figure 3 shows
Performance 360 dimensions (Figure 3a),
line manager friendly Competency Potential
outputs that predict these dimensions
(Figure 3b) and the matched model coun-
terparts in the Wave Styles questionnaire
(Figure 3c) that enables expert users ‘Deep
Dive’ analysis of Facet Ranges, Normative-
Ipsative Splits and Motive-Talent Splits as
described in Kurz et al. (2008). For example,
the Performance 360 criterion dimension of
Generating Ideas is predicted by the Profes-
sional Styles Competency Potential score
with the same name that in turn is based
primarily on the Inventive styles dimension
combined with four other facets that boost
validity.
MacIver et al. (2006) showed excellent
point-to-point prediction of the 36 Behav-
iour dimensions averaging .21 (.39
corrected) for styles and .25 (.46) for the
‘validity-tuned’ competency potential dimen-
sions.
Understanding abilities in the work place
requires a terminology to assess key capabili-
ties as well as validated links to underlying
The Coaching Psychologist, Vol. 4, No. 1, April 2008 13
Effective integration of 360 degree feedback into the coaching process
14 The Coaching Psychologist, Vol. 4, No. 1, April 2008
Almuth McDowall & Rainer Kurz
Figure 3a: Saville Consulting Wave® Performance 360 Criterion Dimensions with Boss, Self, Peer and Report Markers containing Sten values plotted against
Inventory Rating Scale.
Figure 3b: Saville Consulting Wave® Competency Potential Section with Section Graph and Dimension Sten Scores.
Figure 3c: Saville Consulting Wave® Professional Styles Dimension Stens.
aptitude areas. In the course of the stan-
dardisation of Professional Aptitudes indi-
viduals were asked top rate themselves on a
number of matched ability dimensions.
Table 2 shows a good correspondence
between aptitude predictor and ability crite-
rion data which in turn enables meaningful
gap analysis to identify cognitive strengths or
limitations that may help to understand
performance issues.
Finally, global measures of overall
performance effectiveness were developed
that assess contextual performance inde-
pendently of the Behaviour and Ability
model in Saville Consulting Wave following
on from the validation work of Nyfield et al.
(1995) who pioneered the use of job profi-
ciency and promotability criterion scales.
Kurz and Bartram (2002) built on their
research to create the Great Eight compe-
tencies that are used in the SHL Corporate
Leadership model (Bartram, 2002). Bartram
(2005) showed high validity for the Occupa-
tional Personality Questionnaire in the
prediction of job performance using the
Great Eight to aggregate predictor and crite-
rion scales.
Table 3 shows results of a study where the
performance of N=169 managers who
completed OPQ32i and Professional Styles
were rated on three items reflecting global
aspects of performance related to Expertise,
Accomplishment and Potential. Unit weight
Great Eight composite scores were calcu-
lated and aggregated into trait scores and
competency potential scores for Professional
Styles and Corporate Leadership scores for
OPQ32i. All three total scores were strong at
predicting Demonstrating Potential but
weak for Applying Specialist Expertise where
ability measures may be more potent. Profes-
sional Styles clearly improved on the predic-
tion offered by the well-established OPQ32i
tool in this co-validation sample of
respectable size.
In summary the Saville Consulting Wave
approach enables investigation of disposi-
tional and situational variables to explain the
full complexity of job and contextual
performance in the workplace as viewed
from 360 stakeholder perspectives.
The Coaching Psychologist, Vol. 4, No. 1, April 2008 15
Effective integration of 360 degree feedback into the coaching process
Table 2: Professional Aptitudes test validities against Educational (GCSE Points; N=227),Competency Self-Assessment (N=263) and Overall Performance Self-rating (N=263)
criteria.
Verbal Analysis .50 .16 .30 .15 .16 .20
Numerical Analysis .46 .32 .18 .44 .32 .21
Diagrammatic Analysis .39 .22 .01 .18 .23 .08
GC
SEPo
ints
Thin
king
Com
pet
ency
Work
ing w
ith
Word
s
Work
ing w
ith
Num
ber
s
Work
ing w
ith
Dia
gra
ms
Ove
rall
Perf
orm
ance
Using 360 Feedback Profiles inCoaching: Case Study ‘Jo’We would now like to guide our reader on
how to use such feedback ratings as part of a
coaching session by discussing the excerpt
depicted in Figure 3.
At the outset, a coach using 360 needs to
undertake ‘a priori investigations’ about
which 360 dimensions or scales are relevant
to the coachee’s job. Such information
should ideally be based on some sort of job
analysis data or at the very least a consulta-
tion with key people in the coachee’s organ-
isation. This process can be much facilitated
through online tools that allow job profiling
(see Table 1 for the 360 effectiveness check).
Once this has been established, interpre-
tation of the profile and differing ratings can
be done in the light of the role. The focal
individual here is ‘Jo’ who is a senior project
manager in a start-up business unit of a
major technology company. Importance 360
feedback suggested that the focus of this role
is on effective management of the task,
rather than strategic innovation.
The data presented on the right-hand-
side of Figure 3a is based on ratings by the
feedback recipient and nominated others on
the ‘Creating Innovation’ Section in the
‘Solving Problems’ cluster that corresponds
to the ‘Thought’ cluster in the Styles model.
For ‘Generating Ideas’ self and boss ratings
are both in the Fairly Ineffective band. Peer
and report ratings vary but aggregate ratings
are on average considerably higher. Several
potential interpretations are possible and
these would need to be corroborated in a
feedback session.
First, the coach needs to go back to the
earlier investigation of whether this partic-
ular aspect is core to the feedback recipient’s
job, and thus needs to be addressed at all, or
with varying degrees of priority. If the Gener-
ating Ideas dimension was central to the role
and a coach-facilitated feedback session it
would be of paramount importance to get to
the bottom of the differences across the four
rater groups. In the present context, this is
less of an issue as Jo’s current role is to chase
highly imaginative technologists to complete
their projects to externally agreed deadlines.
It is possible that there are different
possible explanations for the patterns in the
ratings, so it is important that the coach
keeps an open mind and explores various
alternative ‘hypotheses’. Here, it is tenable
that both the manager and Jo see Gener-
ating Ideas as unrelated to the focal job role,
and both view this as ‘In Agreement – Poor’.
Feedback interview questions might include
‘In what way is coming up with new ideas
core to your job?’, ‘Could you give me some
concrete examples where you were expected
to generate ideas?’, ‘What stops you from
16 The Coaching Psychologist, Vol. 4, No. 1, April 2008
Almuth McDowall & Rainer Kurz
Criterion Applying Accomplishing Demonstrating Global
Specialist Objectives Potential Performance
Expertise (sum of
3 items)
Predictor
OPQ32i
Corporate Leadership Score-.04 .07 .17* .09
Professional Styles
Trait Score .04 .24** .22** .21**
Professional Styles
Competency Potential Score.08 .26** .27** .26**
Table 3: Co-validation Prediction of Global Performance through OccupationalPersonality Questionnaire and Professional Styles ‘Great 8’ Scores (N=169).
coming up with new ideas, and what helps?’
In contrast, peer and report raters might
find Jo an inspirational source of ideas on
how to manage projects more effectively, and
thus perceive her performance as fairly effec-
tive, hence the difference in ratings. The
other raters have then actually observed rele-
vant and effective behaviours, which are not
salient to Jo or her manager as they might be
focused on a more narrow understanding of
this behavioural domain or benchmark her
against highly imaginative technologists. In
this instance, it can be helpful to bring
circular questions into the 360 interview
(asking how behaviours might be perceived
by others, and what the effect on others
might be), to encourage Jo to widen her own
perspective.
An alternative hypothesis is that Jo may
be an ‘Under-rater’ who self-rates perform-
ance as lower than others, but actually
performs effectively. This could be coupled
with a lack of self-confidence, which might
be central to understanding the boss’s
ratings which are also low. If there is a lack of
confidence, effective behaviours might not
be salient to others such as the boss, or
underplayed by the feedback recipient. In
this instance, coaches would need to take
care to question Jo particularly sensitively,
first focusing on effectiveness in the work-
place, then shifting the focus to how confi-
dence and self-esteem could effectively be
nurtured and supported. The coach could
explore further by bringing in motives and
talent from a styles assessment.
It is possible that an observed difference
between these could provide the coach with
further prompts as to whether an individual
is realising their full potential with regards to
Generating Ideas but also how Jo might be
adapting to pressures in the workplace. The
gap analysis between potential and perform-
ance 360 rating will help to establish whether
the person or the environment/situations
should be developed, and how difficult such
a change is likely to be.
Taking the last example of Jo holding a
senior project management role in a tech-
nology consultancy, one hypothesis for
exploration holds that Jo is capable of
creating innovation, but due to the pressures
of the consultancy environment performs at
less than optimal level. When this styles
assessment is used alongside the Perform-
ance 360 report it allows an understanding
of the degree to which an individual is
fulfilling their potential in performance
terms. This further exploration can generate
greater insight into potential reasons for the
differences in the effectiveness ratings on
Performance 360.
If Jo were to be considered for promo-
tion to a more senior role we should look at
her styles results that suggest moderate
competency potential for Generating Ideas,
and a rather strong inclination to generate
original ideas. In a more challenging role
where creativity is called for Jo has fairly high
potential for success. Aptitude assessment
results of Jo actually were above average for
five of the six areas.
Returning back to our earlier proposi-
tion that self-regulation is key to under-
standing 360 effectiveness, the coach may
need to decide how best to facilitate this. For
instance, it can be important in which order
psychometrics are discussed with the
coachee. As 360 output is typically conceptu-
alized as a criterion measure, it can be
helpful to present this to coaches first to
help in the formulation of discrete goals,
before any ‘predictor evidence’ (such as apti-
tudes, motives and talents) is used in subse-
quent sessions to work out exactly how such
goals could be achieved.
ConclusionWe conclude that 360 degree feedback meas-
ures make an effective contribution to the
coaching process, as differences in ratings
provide both the coachee and the coach with
valuable information about levels of effective
performance at work. It is essential however,
that the tool itself has undergone thorough
psychometric validation and thus offers reli-
able and robust information. Where this is
achieved 360 data provides the coach with a
The Coaching Psychologist, Vol. 4, No. 1, April 2008 17
Effective integration of 360 degree feedback into the coaching process
sound model for exploring accuracy of infor-
mation and levels of self-insight and confi-
dence with their coachees. It is essential that
coach and coachee understand this in the
context of the requirements of a particular
role, and have some indication of how focal
individuals can realise their potential. In
summary, 360 may provide a very finely
grained measure of performance which
provides structure and robustness to the
coaching process, and can be used to even
greater effectiveness when coupled with
aptitude, motives and talent measures of
potential.
CorrespondenceAlmuth McDowall
University of Surrey,
Psychology Department,
Faculty of Arts and Human Sciences,
Guildford GU2 7HX.
E-mail: [email protected]
Rainer Kurz
Saville Consulting UK Ltd,
Harley House, 94 Hare Lane,
Claygate,
Surrey KT10 0RB.
E-mail: [email protected]
Useful web-linkswww.savilleconsulting.com
18 The Coaching Psychologist, Vol. 4, No. 1, April 2008
Almuth McDowall & Rainer Kurz
References
Alimo-Metcalfe, B. & Alban-Metcalfe, R.J. (2001).The development of a new transformationalleadership questionnaire. Journal of Occupational
& Organisational Psychology, 74(1), 1–27. Alimo-Metcalfe, B. & Alban-Metcalfe, R.J. (2008). The
impact of engaging leadership on organisational
performance. Paper presented at the BritishPsychological Society Division of OccupationalPsychology Conference, Stratford-upon-Avon,January.
Antonioni, D. (1996). Designing an effective 360-degree appraisal process. Organisational
Dynamics, 2, 24–38.Atwater, L. & Brett, J.F. (2005). Antecedents and
consequences of reactions to developmental 360feedback. Journal of Vocational Behaviour, 66,532–548.
Atwater, L. & Brett J.F. (2006). Feedback format:Does it influence manager’s reactions tofeedback? Journal of Occupational and
Organisational Psychology, 79, 517–532.Atwater, L., Roush, P. & Fischthal, A. (1995). The
influence of upward feedback on self andfollower ratings of leadership. Personnel
Psychology, 48, 35–60.Avolio, B., Bass, B.M & Jung, D.I. (1995). MLQ
Multifactor Leadership Questionnaire. RedwoodCity, CA: Mind Garden.
Bass, B.M. & Avolio, B.J. (1990). Transformational
Leadership Development: Manual for the Multifactor
Leadership Questionnaire. Palo Alto, CA:Consulting Psychologists Press.
Bass, B.M., Avolio, B.J. & Atwater, L. (1996). Thetransformational and transactional leadership ofmen and women, International Review of Applied
Psychology, 45, 5–34. Bass, B.M. & Yammarino, F.J. (1991). Conguence of
self and others’ leadership ratings of navalofficers for understanding successfulperformance. Journal of Applied Psychology, 40,437–454.
Bailey, C. & Austin M. (2006). 360 degree feedbackand developmental outcomes: The role offeedback characteristics, self-efficacy andimportance of feedback dimensions to focalmanagers’ current role. International Journal of
Selection and Assessment, 14, 51–66.Bailey, C. & Fletcher, C. (2002). The impact of
multiple source feedback on managementdevelopment: Findings from a longitudinal study.Journal of Organisational Behavior, 23(7), 853–867.
Bartram, D. (2002). The SHL Corporate Leadership
Model. SHL Research White Paper. Thames Ditton:SHL Group plc.
Bartram, D. (2005). The Great Eight competencies:A criterion-centric approach to validation.Journal of Applied Psychology, 90, 1185–1203.
Carver, C. & Scheier, M. (1981). Attention and self-
regulation: A control theory approach to human
behaviour. New York: Springer.Fletcher, C. & Baldry, C. (1999). Multi-source
feedback systems: A research perspective. In C.L.Cooper & I.T. Robertson (Eds.), International
Review of Industrial and Organisational Psychology.
Volume 14. New York/London: Wiley.
The Coaching Psychologist, Vol. 4, No. 1, April 2008 19
Effective integration of 360 degree feedback into the coaching process
Fletcher, C., Baldry, C. & Cunningham-Snell, N.(1998). The psychometric properties of 360-degree feedback: An empirical study and acautionary tale. International Journal of Selection
and Assessment, 6, 19–33.Gregura, G.J., Ford, J.M. & Brutus, S. (2003).
Manager attention to multi-source feedback.Journal of Management Development, 22, 345–361.
Hazucha, J.F., Hezlett, S.A. & Schneider, R.J. (1993).The impact of 360-degree feedback onmanagement skills development. Human Resource
Management Journal, 32, 325–351. Jones, S. (2007). ‘Staff appraisals: Waste of time.’
Retrieved 12 December, 2007, from:http://news.bbc.co.uk/1/hi/business/7126971.stm
Kluger, A.N. & DeNisi, A. (1996). The effects offeedback interventions on performance:Historical review, a meta-analysis and apreliminary feedback intervention theory.Psychological Bulletin, 119, 254–284.
Kurz, R. & Bartram D. (2002). Competency andindividual performance: Modelling the world ofwork. In I.T. Robertson, M. Callinan & D. Bartram (Eds.), Organizational effectiveness:
The role of psychology. John Wiley & Sons.Kurz, R., Saville P. & MacIver R. (2008). Coaching
using Saville Consulting Wave. In J. Passmore(Ed.), Psychometrics in coaching. London: KoganPage.
Latham, G.P. (2006). Work motivation: History, theory,
research and practice. London: Sage.Mabey, C. (2001). Closing the circle: Participant views
of a 360-degree feedback programme. Human
Resource Management Journal, 11(1), 41–53.MacIver, R., Saville, P., Kurz, R., Mitchener, A.,
Mariscal, K., Parry, G., Becker, S., Saville, W.,O’Connor, K., Patterson, R. & Oxley, H. (2006).Making Waves – Saville Consulting Wave Stylesquestionnaires. Selection & Development Review,
22(2), 17–23.
Marsh, H.W. & Roche, L.A. (1997). Making students’evaluations of teaching effectiveness effective –The critical issues of validity, bias, and utility.American Psychologist, 52(11), 1187–1197.
Maurer, T.J., Mitchell, D.R.D. & Barbeite, F.G. (2002).Predictors of attitudes toward a 360-degreefeedback system and involvement in post-feedback management development activity.Journal of Occupational and Organisational
Psychology, 75(1), 87–107.McDowall, A. (2008). Using feedback in coaching.
In J. Passmore (Ed.), Psychometrics in coaching.
London: Kogan Page.McDowall, A & Kurz, R. (2007). Making the most of
psychometric profiles – effective integration intothe coaching process. International Coaching
Psychology Review, 2(3), 299–309.Nyfield, G., Gibbons, P.J., Baron, H. & Robertson, I.
(1995). The cross cultural validity of management
assessment methods. Paper presented at the 10thAnnual SIOP Conference, Orlando, May.
Smither, W., London, M. & Reilly, R.R. (2005). Doesperformance improve following multisourcefeedback? A theoretical model, meta-analysis,and review of empirical findings. Personnel
Psychology, 58, 33–52.Smither, J.W., London W., Flautt, R., Vargas, Y. &
Kucine, I. (2003). Can working with an executivecoach improve multi-source feedback ratingsover time? Personnel Psychology, 56(1), 23–44.
Yammarino, F. & Atwater, L. (1993). Understandingself-perception accuracy: Implications for humanresource management. Human Resource
Management, 32, 231–247.Yammarino, F. & Atwater, L. (1997). Do managers see
themselves as others see them? Organisational
Dynamics, 25, 35–44.
Coachingat Workwww.cipd.co.uk/coachingatwork
“An inspiring
and thoughtprovokingresource”
KAREN BOANASBusiness Consultant,
Canada Life
Coaching at Work is only available on subscription.
For more details and to subscribe at this special rate visit
www.cipd.co.uk/coachingatwork or call 0844 3221274.
Please quote SGCP2008**available until December 2008
Coaching at Work has established itself as the must-read publication for everyone involved in coaching. As a SGCP member you can now subscribe at a special introductory rate that is HALF the normal subscription cost.
Both inspiring and thought-provoking, the bi-monthly magazine addresses all the important issues that concern today’s coaching professionals whilst the online resource provides access to extensive reference material, and allows you to give your views on the issues raised.
“ Outstanding integration of coaching practice, theory and research - read it!”
DR ANTHONY GRANTSydney University
20 The Coaching Psychologist, Vol. 4, No. 1, April 2008
Development of the multimodalapproach
THE MULTIMODAL APPROACH was
developed by a psychologist, Arnold
Lazarus (1989, 1997) in the 1970s. He
believed that no one system of therapy could
provide a complete understanding of either
human development or condition. By
following-up counselling clients who had
received behaviour therapy he found that
many of them had relapsed. He noted that
when clients had used both behaviour and
cognitive techniques more durable
outcomes were obtained. Hence he started
using a broad range of cognitive-behavioural
techniques applied systematically and finally
this led to the development of Multimodal
Therapy which focuses on seven discrete but
interactive dimensions or modalities which
encompass all aspects of human personality
(Lazarus 1971, 1973).
The multimodal approach has been
adapted and applied to executive coaching
(Richards, 1999), personal/life coaching,
stress management and health coaching
(Palmer, 2003; Palmer et al., 2003; Palmer &
Gyllensten, 2008). It pre-dates cognitive-
behavioural models such as SPACE (Edgerton
& Palmer, 2005). Successive surveys of
coaching psychologists have found that about
10 per cent of respondents use multimodal
coaching (see Whybrow & Palmer, 2006a, b;
Palmer & Whybrow, 2004, 2007).
Basic theory and practiceThe Multimodal approach is underpinned
by a broad social and cognitive learning
theory (Bandura, 1977, 1986), although it
also uses group and communications theory
and general systems theory as necessary
(Lazarus, 1989). It assumes that people are:
‘essentially biological organisms
(neurophysiological/biochemical entities) who
behave (act and react), emote (experience
affective responses), sense (respond to olfactory,
tactile, gustatory, visual and auditory
stimuli), imagine (conjure up sights, sounds
and other events in the mind’s eye), think (hold
beliefs, opinions, attitudes and values), and
interact with one another (tolerate, enjoy or
suffer in various interpersonal relationships).
These dimensions of personality are usually
known by the acronym, BASIC I.D. derived
from the first letters of each modality, namely
Behaviour, Affect, Sensations, Images,
Cognitions, Interpersonal and Drugs/biology
(Palmer, 2006, pp.322–323).
The BASIC I.D. modalities often interact
with each other and may exist in a state of
reciprocal transaction. For example, an
Image in one’s mind of performing badly at
work could trigger a Cognition ‘I’m a failure’
which could subsequently lead to an Affect
(emotion) of depression.
Coachees presenting a number of issues
or problems for coaching may need a range
of strategies and techniques to deal with
The Coaching Psychologist, Vol. 4, No. 1, April 2008 21
© The British Psychological Society – ISSN: 1748–1104
Multimodal coaching and its applicationto workplace, life and health coaching
Stephen Palmer
This article highlights how the multimodal approach (Lazarus, 1989) has been adapted to the field of
coaching and coaching psychology. It covers the basic theories underpinning the multimodal approach and
illustrates the link between the theory and practice. Key multimodal strategies are covered including
modality profiles, structural profiles, tracking and bridging.
Keywords: multimodal coaching and therapy, workplace coaching, health coaching, Arnold
Lazarus, modality profiles, structural profiles, tracking, bridging.