R squared effect-size measures and overlap between directand indirect effect in mediation analysis

Peter de Heus

Published online: 19 August 2011# The Author(s) 2011. This article is published with open access at Springerlink.com

Abstract In a recent article in this journal (Fairchild,MacKinnon, Taborga & Taylor, 2009), a method wasdescribed for computing the variance accounted for bythe direct effect and the indirect effect in mediationanalysis. However, application of this method leads tocounterintuitive results, most notably that in somesituations in which the direct effect is much strongerthan the indirect effect, the latter appears to explain muchmore variance than the former. The explanation for this isthat the Fairchild et al. method handles the stronginterdependence of the direct and indirect effect in away that assigns all overlap variance to the indirecteffect. Two approaches for handling this overlap arediscussed, but none of them is without disadvantages.

Keywords Effect-size . Mediation analysis

In the 25 years since the seminal article by Baron & Kenny(1986), mediation analysis has become an indispensablepart of the statistical toolkit for researchers in the socialsciences. Although this has given rise to an extensiveliterature (for an overview, see MacKinnon, 2008), measuresof effect size in mediation analysis appear to have laggedbehind. In a recent article, Fairchild et al. (2009) discussedthe limitations of previous effect size measures, andproposed R squared effect-size measures as a viablealternative. In the present study, I will show that these Rsquared measures suffer from a serious problem, an

asymmetric treatment of the direct and indirect effectthat can not be justified on normative grounds. I willindicate the source of this problem (assigning alloverlap between direct and indirect effect to the indirecteffect), and provide some solutions that may providesome help to researchers in deciding how to handle thisinterdependence in variance explained by the direct andindirect effect.

R squared measures for the direct and indirect effect

In order to keep the conceptual points of the present articleas simple and clear as possible, I will limit myself to thethree-variable case with one dependent variable (Y), oneindependent variable (X), and one mediator (M). Also forsimplicity (because it removes standard deviations from allformulas), my description will be in terms of standardizedregression weights (betas).

The total, the direct, and the indirect effect

The total effect (βtot) of X on Y is given by βYX , theregression weight in a regression analysis in which Y ispredicted by X only. The direct effect (βdir) of X on Y isgiven by βYX.M , the beta of X in a regression analysis inwhich Y is predicted by both X and M. The indirect effect(βind) of X on Y via M is given by the product βMX βYM.X,in which βMX is the beta in a regression in which M ispredicted by X only, while βYM.X is the beta of M in aregression analysis in which Y is predicted by both X andM (Fig. 1).

Following these definitions and using standardequations from ordinary least squares regression analysis(e.g. Cohen, Cohen, West and Aiken, 2003), the total,

P. de Heus (*)Department of Psychology, University of Leiden,Wassenaarseweg 52,2333 AK Leiden, the Netherlandse-mail: deheus1@fsw.leidenuniv.nl

Behav Res (2012) 44:213–221DOI 10.3758/s13428-011-0141-5

direct, and indirect effect can be computed from theintercorrelations of Y, X, and M as follows.

btot ¼ bYX ¼ rYX ð1aÞ

bdir ¼ bYX :M ¼rYX � rMX rYM

1� r2MXð1bÞ

bind ¼ bMX bYM :X ¼rMX rYM � rMX rYXð Þ

1� r2MXð1cÞ

As described in almost all texts and articles on mediationanalysis (e.g. MacKinnon, 2008), and as can be verified byadding the right terms of Eqs. 1b and 1c, the total effect isthe sum of the direct effect and the indirect effect.

btot ¼ bdir þ bind ð2Þ

Effect size

Perhaps the most commonly used measure of effect sizein mediation analysis is the proportion mediated (PM:Alwin and Hauser, 1975; MacKinnon and Dwyer, 1993),which simply gives the (direct or indirect) effect as aproportion of the total effect: PMdir = βdir / βtot andPMind = βind / βtot . As demonstrated by simulationstudies (MacKinnon, Warsi and Dwyer, 1995), theproportion mediated suffers from instability and bias insmall samples, so it needs large samples (N>500) toperform well. In the light of these limitations, investigat-ing alternative effect size measures such as R squaredmeasures seems to be a worthwhile effort (for a morecomplete overview of effect size measures in mediation,see Preacher and Kelley, 2011).

Given the previous definitions of the total, the direct, andthe indirect effect, how much variance is explained by

each? The total amount of Y variance explained by X (i.e.the variance explained by the total effect), R2tot, is equal tor2YX , the squared correlation between Y and X.

Everything described until this point is generallyaccepted and completely uncontroversial. What’s new inFairchild et al. (2009) is a procedure to decompose thistotal amount of variance explained into two parts, one forthe direct effect and one for the indirect effect (Fig. 2).According to these authors, R2dir, the variance explained bythe direct effect, corresponds to the Y variance that isexplained by X but not by M, whereas R2ind , the varianceexplained by the indirect effect, corresponds to thevariance that is shared by Y, X and M together. Definedthis way, the variance explained by the direct effectcorresponds to the squared semipartial correlationr2Y ðX :MÞ, and the variance explained by the indirect effectis the difference between total and direct effect varianceexplained. Of these three measures, the first two can onlybe positive or zero, but as noted by Fairchild et al. (2009),it is perfectly possible that the third, R2ind , is negative.

1

R2tot ¼ r2YX ð3aÞ

R2dir ¼ r2Y X :Mð Þ ¼rYX � rYMrMXffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

1� r2MX

q

0

B@

1

CA

2

ð3bÞ

R2ind ¼ R2tot � R2dir ¼ r2YX � r2Y X :Mð Þ ð3cÞ

At first sight, this way of decomposing the total varianceexplained into a direct and an indirect part has veryattractive features in comparison to proportion mediated.First, thinking in terms of variance explained is verycommon among social scientists (so common, actually, thatif one uses the proportion mediated instead, one very oftenhas to explain to one’s colleagues that this proportion doesnot refer to variance explained). Second, the Fairchild et al.decomposition of variance explained allows for easycomputation, using only well-known statistical concepts.Third, as shown by simulation studies, the proposedmeasures have nice statistical properties like stability andacceptable bias levels (Fairchild et al., 2009). Fourth, and

1 Actually, Fairchild et al. (2009) used commonality analysis (Mood,1969; Seibold and McPhee, 1979) for the decomposition of total effectvariance. This has the advantage of allowing generalization to caseswith two or more independent variables and/or mediators, but is notnecessary for the more conceptual discussion of the single predictor -single mediator case to which the present article is limited, so I willnot follow them in this regard.

X

YX

M

Y

Total effect

Direct and indirect effect

ßYX

ßMX ßYM.X

ßYX.M

Fig. 1 The total, the direct, and the indirect effect

214 Behav Res (2012) 44:213–221

perhaps most important, decomposing Y variance in thisway appears to be intuitively clear. The conceptualization ofvariance explained by the direct effect is completely inaccordance with how according to all textbooks (e.g. Cohenet al., 2003) we should compute the unique varianceexplained by a predictor in regression analysis. In addition,the idea that variance explained by the indirect effect is thevariance shared by all three variables together (predictor,mediator, and dependent variable) also comes as a verynatural one. If the part of the common variance of X and Ythat is not shared by M refers to the direct effect, what couldbe more natural than assuming that the remaining part ofthe common variance of X and Y, the part that is also sharedby M, represents the indirect effect?

In a recent review of effect size measures in mediation,Preacher and Kelley (2011, p. 100) concluded that theFairchild et al. measure “… has many of the characteristicsof a good effect size measure: (a) It increases as the indirecteffect approaches the total effect and so conveys informa-tion useful in judging practical importance; (b) it does notdepend on sample size; and (c) it is possible to form aconfidence interval for the population value.” On thenegative side, these authors noted that because of thepossibility of negative values, the R squared measure is nottechnically a proportion of variance, which limits itsusefulness. The final verdict was that “… it may haveheuristic value in certain situations” (Preacher and Kelley,2011, p. 100). In the next section, I will address acompletely different problem with the R squared measure,which is not dependent on negative values (although it maybe aggravated by them).

Problems

Problems started with a counterintuitive example, thatproved to be a special case of a more general problem.

A counterintuitive example

Applying the R squared effect-size to an example in acourse that I was teaching led to very counterintuitiveresults. In this course example (N = 85) about the effectof social support at work (X) on depression (Y) withactive coping (M) as mediator, zero-order correlationswere rYX = −.336, rMX = .345, and rYM = −.391, leading tothe following standardized estimates for the total,direct and indirect effects: ßtot = −.336; ßdir = −.228;ßind = −.108. In terms of proportion mediated, the directeffect (PMdir = .679) was more than twice as strong as theindirect effect (PMind = .321). Given such a large differencein favor of the direct effect, one would definitely expectthat the direct effect would explain more variance of thedependent variable than the indirect effect. However,computing the R squared measures from Fairchild et al.(2009) led to R2dir = .046 and R

2ind = .067, so in terms of

variance explained the indirect effect appeared to bemuch stronger than the direct effect. This reversal ofeffect size from one effect being twice as large than theother in terms of proportion mediated, but much smallerin terms of R squared effect-size is by no means trivial. Atthe time I could only say to my students that I did notunderstand this discrepancy between proportion mediatedand proportion of variance explained, but that I would tryto find out. The present article is a direct consequence ofthat promise.

Asymmetry of direct and indirect effect

A first step in clarification is to be more specific aboutwhat we should expect from an R squared effect sizemeasure. To my opinion, one should expect an R squaredmeasure to be symmetric in the sense that a direct andindirect effect of the same magnitude (ßc) should lead tothe same amount of variance explained. After all, if a onestandard deviation gain in X leads to an average gain of ßcstandard deviations in Y, there is no reason why it shouldmake a difference for variance explained whether this gainis due to the direct or to the indirect effect. Since thechange in Y as a function of X is identical in both cases,and since change in Y is all that matters (or should matter)for Y variance explained, one would expect amount ofvariance explained to be identical.

If this reasoning is correct, one would expect thatexchanging the values for the direct and indirect effectwould lead to a simple exchange of the R square changemeasures. In order to check this, the course example wasmodified. By keeping rYX and rMX at −.336 and .345respectively, but changing rYM to −.697, a reversal of theoriginal values for the direct and indirect effect wasaccomplished with ßdir and ßind equal to −.108 and −.228

X

M

Y

Rdir2

Rmed2

Fig. 2 Partitioning of total variance explained over direct, andindirect effect according to Fairchild et al. (2009)

Behav Res (2012) 44:213–221 215

respectively. However, the R squared measures in thismodified example were not the reverse of those in theprevious analysis (.046 and .067). Instead, the actualvalues were R2dir = .010 and R

2ind = .103, so whereas in

terms of proportion mediated the indirect effect is nowabout twice as strong as the direct effect, in terms ofvariance explained it is more than ten times as strong. In alast example, the direct and indirect effect were madeidentical. Keeping rYX and rMX at −.336 and .345respectively, but changing rYM to −.545, led to identicalbetas for the direct and indirect effect, ßdir = ßind = −.168,but the obtained R squared change measures were R2dir =.025 and R2ind = .088. At least in these three examples, thegeneral pattern seems to be that the R squared measures byFairchild et al. (2009) are not symmetric, but stronglybiased in favor of the indirect effect.

In hindsight, even without these examples we mighthave known that the R squared measures for the direct andindirect effect can not always be symmetric, because asmentioned by Fairchild et al. (2009), R2dir will always bepositive or zero, whereas R2ind may also be negative. WhenR2ind is negative, exchanging the betas for the direct andindirect effect can never lead to exchanging their R squaredmeasures, because R2dir can not become negative. Thegeneral question is how it is possible that identical directand indirect effects, with identical effects on Y, can becompletely different in terms of R squared effect size?

Explanation and solutions

Direct and indirect effect are not independent

A look at the equations and illustrations in the Fairchildet al. (2009) article leaves one with the impression thattotal variance explained can unequivocally be dividedover the direct and the indirect effect, as if those twoeffects are completely independent. As will be arguedbelow, this is not true. The key to understanding thedivergent results for proportion mediated versus varianceexplained measures and for the asymmetry of the directand indirect effect, is that the direct and indirect effectare heavily interdependent.

Although the mediator M is crucial for estimation of thedirect and indirect effect, once the relevant path coefficientshave been estimated, both direct and indirect effect are afunction of the independent variable X only. For computingvariance explained, the situation is the same as if we had asingle predictor with a single regression weight that forsome reason can be split into two parts. For example, if wepredict income (Y) from the number of hours worked (X)for a group of laborers with the same hourly wages and thesame hourly bonus, we are formally in the same situation as

with the direct and indirect effect. Here too we have anoverall effect that is just the sum of two separate effects(regression weights) for the same independent variable:btot ¼ bwages þ bbonus. If we try to answer the question howmuch variance of income is explained by wages and howmuch by bonus, formally our problem is exactly the sameas when we try to predict how much of the variance of thetotal effect is due to the direct effect and how much to theindirect effect.

The fact that (once ßdir and ßind have been estimated) thedirect and indirect effect both are a function of the samepredictor and nothing else, makes them heavily interde-pendent. In fact, they are perfectly correlated, becauseeach individual’s predicted gain due to the indirect effectwill always be βind / βdir times his or her gain due to thedirect effect. The consequences of this become clear whenwe write the total proportion of variance explained as afunction of the direct and indirect effect:

R2tot ¼ b2tot ¼ bdir þ bindð Þ2 ¼ b2dir þ b2ind þ 2bdirbind ð4ÞAs can be seen in eq. 4, total variance explained is

now divided into three parts, of which the first two can beunequivocally related to the direct (b2dir) or indirect (b

2ind)

effect, but the third (2bdirbind) is a joint part, for whichascription to either direct or indirect effect is notstraightforward. This joint part may be positive ornegative, depending on whether ßdir and ßind are of sameor opposite sign.

Two ways of dividing variance over direct and indirecteffect

Given these three variance components, there are at leasttwo defensible ways to divide variance explained over thedirect and indirect effect, but neither of them is withoutdisadvantages.

Unique approach Perhaps the most “natural” solution isjust taking b2dir and b

2ind as variance explained for the direct

and indirect effect respectively.

R2dirðUÞ ¼ b2dir ð5aÞ

R2indðUÞ ¼ b2ind ð5bÞ

These squared betas refer to the proportion of varianceexplained by each effect if it were completely on its own(i.e. if the other effect were zero). This unique approachis roughly comparable with the Type III sum of squaresapproach of partitioning variance (also known as theunique or regression approach: Stevens, 2007) inANOVA. In our unmodified example, this leads to

216 Behav Res (2012) 44:213–221

proportions of variance explained of (−.228)2 = .052 and(−.108)2 = .012 for the direct and indirect effect respec-tively, while 2 x (−.228) x (−.108) = .049 is not uniquelyexplained.

An advantage of this unique approach is that it has aclear causal interpretation: The proportion of varianceexplained by the effect if the other effect did not exist orwas blocked somehow. Furthermore, computing variancesexplained in this way will never lead to negative variancesor to order reversals between proportion mediated andvariance explained as in the examples, and more generally,will not be asymmetric in its treatment of the direct and theindirect effect. Disadvantage is that this way of computingvariance completely ignores the joint part 2bdirbind , whichmay add or subtract a large amount of variance to or fromthe total effect.2

Hierarchical approach If we have reason to view one effect(which I will call the primary effect) as more fundamentalor more important than the other (the secondary effect), wecould use a hierarchical approach. For the primary effect weuse the variance explained by the effect on its own as in theunique approach, and for the secondary effect we use theadditional variance explained over and above what wasexplained by the primary effect.

R2primaryðHÞ ¼ b2primary ð6aÞ

R2secondary Hð Þ ¼ b2secondary þ 2bprimarybsecondary ð6bÞ

Because the secondary effect gets the joint (overlap)variance, its variance explained may be negative whendirect and indirect effect are of opposite sign. Thishierarchical approach is roughly comparable with the TypeI sum of squares approach (also called the hierarchicalapproach) of partitioning variance in ANOVA (Stevens,2007). If we take the direct effect as primary in ourunmodified example, we get a slightly different version ofour original, counterintuitive results: .052 and .061 for thedirect and indirect effect respectively. If we take the indirecteffect as primary, we get very different values: .101 (direct)and .012 (indirect).

The main advantages of the hierarchical approach arethat all variance explained is neatly and uniquely dividedover the direct and the indirect effect, and that both effectshave a clear interpretation, namely variance explained bythe effect if it were on its own (primary), and additionalvariance explained over and above what was explained bythe primary effect (secondary). However, these advantages

only apply when we have good reasons for decidingwhich effect should be treated as primary, and which assecondary. An example of such good reasons is if one ofthe effects is the intended one and stronger than the othereffect, which is supposed to represent only relativelyminor side effects. One might think that the direct effectis more suitable for the role of primary effect, but this isnot necessarily so. For example, if a psychotherapy isintended to reduce psychological complaints by encouragingself-efficacy, it makes sense to treat the indirect effect(from therapy via self-efficacy to psychological com-plaints) as primary, and the direct effect (which inaddition to a possible “real” direct effect also representsall indirect effects for which the possible mediators havenot been measured) as secondary. Furthermore, as will beargued in next section, when the direct and indirect effectare of opposite sign, it makes sense to treat the effectwith the largest absolute size as primary. The maindisadvantage of the hierarchical approach is that oftenthere are no decisive theoretical or statistical argumentsfor deciding which effect is the primary one.

Negative variance explained

An often noted problem in texts on regression analysis (e.g.Cohen et al., 2003) in regard to graphical displays ofvariance explained like Fig. 1, is that the value of theoverlap area of all three variables can be negative, whereasboth area and variance are squared entities, which shouldalways be equal or larger than zero. Although thispossibility of negative variance/area is not a problemfor regression analysis, since the reasons for it are well-understood, it ruins the area-represents-variance metaphorthat gives these kinds of figures their heuristic value.

Here I will present a different graphical approach, thatmay provide a more helpful illustration of negative varianceexplained, using squares instead of circles and taking thesigns of the two betas on which the total effect is based intoaccount (Fig. 3). In what follows, the two betas will just becalled ß1 and ß2, because for the present purpose it does notmatter which one represents the direct or the indirect effect.There are two interesting cases here, dependent on whetherthe two betas have the same sign (case 1) or different signs(case 2).3 Only in case 2 will we be confronted withnegative variance explained.

Case 1 (same signs) Because total variance explained isequal to b2tot ¼ b1 þ b2ð Þ2, it is given by the area of thelarge square with sides ß1 + ß2 in Fig. 3a. The two squares

3 Of course, there is also the possibility that one or both betas are zero,but about this I have nothing of interest to say.

2 The joint part 2bdirbind can take values between � b2dir þ b2ind� �

(if bdir ¼ �bind), and b2dir þ b2ind (if bdir ¼ bind).

Behav Res (2012) 44:213–221 217

in the lower left and upper right of the large square, withareas b21 and b

22 respectively, give the variance explained for

effect 1 and 2 for the (here clearly hypothetical) situation inwhich the other effect were zero. The two rectangles in theupper left and lower right within the large square, both witharea ß1 ß2, together represent the overlap part 2ß1 ß2, whichwe can assign to neither effect (unique approach), or to thesecondary effect only (hierarchical approach).

Case 2 (different signs) If direct and indirect effect haveopposite signs, the effect with largest absolute value willalways explain more variance than the total effect. This isillustrated in Fig. 3b, which looks very much like Fig. 3a,but now the large square has area b21 and refers to thevariance explained by the largest of the two effects. Thevariance explained by the total effect is represented by thesmaller square with area b1 þ b2ð Þ2 in the lower left withinthe large square. Here a hierarchical interpretation with thelargest effect as primary is the most natural way to describewhat happens to variance explained. It goes like this. Theproportion of variance explained by effect 1 on its own (i.e.if ß2 were zero) would be b

21 (the large square), but because

effect 2 makes ßtot smaller than ß1, total variance explainedis represented by the smaller white square within the largesquare. One way of describing how to get the area of thissmall square from the area of the large square is that wehave to subtract from the area of the large square (b21) theareas of the two rectangles ABCD and DEFG (both �b1b2),but because the square CDEH is part of both rectangles, itsarea (b22) is subtracted twice, so to compensate for that, wehave to add this area once. This leads to the subtractionb21 � �2b1b2 � b22

� � ¼ b21 þ 2b1b2 þ b22, which neat lyequals b2tot. In this graphical interpretation of varianceexplained, there is never negative area, but only subtractionof one positive area from another.

Incidentally, this interpretation only works well if we takethe largest effect as primary in our hierarchical approach.Algebraically, it makes no difference whether we start from b21or from b22, but geometrically, I do not see an intuitivelyrevealing picture that clarifies how to get from the smallsquare with area b22 to the larger square with area b1 þ b2ð Þ2.Putting into words how exactly to get from so much varianceexplained by one effect to even more variance explained by atotal effect that is in the opposite direction, is not easy either.Therefore, if the direct and indirect effect are of oppositesign, the most intuitively appealing approach is hierarchical,with the largest effect as primary.

Comparison with the Fairchild et al. approach

The method presented by Fairchild et al. (2009) can be seenas a slightly modified version of the hierarchical approachin the present article. Their R squared measure for the directeffect, the squared semipartial correlation (Eq. 3b), isclosely related to the squared beta for the direct effect(which is used in the hierarchical approach when the directeffect is taken as primary), because it follows from Eqs.(1b) and (3b) that:

r2Y X :Mð Þ ¼ 1� r2XM� �

b2dir ð7Þ

The R squared measure for the indirect effect gets all thatremains of the total variance explained after removingdirect effect variance. This means that in the Fairchild et al.approach the direct effect is taken as primary and theindirect effect as secondary. As a consequence, the indirecteffect always gets the joint part 2bdirbind , so it will befavored in comparison to the direct effect when this jointpart is positive, which is when both effects have the same

Case 1. β1 and β2 have same sign Case 2. β1 and β2 have different signs(here both positive) (here β1 positive and |β1| > |β2|)

21 β2β

2β

β

1β

1β

21β

22β

21 ββ2

21 )( ββ

21 β 2ββ

1β

21 β

2β

β

1β

1β

+

+

+

21 ββ +

+

22β2

A

CB

DE

F G

H

2221 ββ −−β− ß

2221 ββ −

−

− ß

Fig. 3 An alternative picture ofthe partitioning of total varianceexplained over the direct andindirect effect (and theiroverlap), without negative area

218 Behav Res (2012) 44:213–221

sign. When the joint part is negative, the indirect effect willbe disfavored, possibly leading to negative varianceexplained. These are the reasons why in our examples (inwhich both effects always had the same sign) the directeffect was much lower than expected. It also explains whythe indirect effect, but not the direct effect can becomenegative.

Modifying the Fairchild et al. approach

As argued previously, the problem in the Fairchild et al.approach is not in the direct effect measure (the squaredsemipartial correlation), which taken on its own makesperfect sense, but in the asymmetry between direct andindirect effect measures. Therefore, instead of the previousdecomposition of equation (4) into different parts, analternative approach is retaining the Fairchild et al. directeffect measure, but changing the indirect effect measure in away that makes it symmetrical with the direct effectmeasure.

This alternative approach goes in two steps. First, the Yvariance that is uniquely explained by M is given by thesquared semipartial correlation r2Y M :Xð Þ. However, not allthis variance can be attributed to the indirect effect, becauseM is only partially explained by X. Second, multiplyingr2Y M :Xð Þ, the proportion of Y variance uniquely explained byM, with r2MX , the proportion of M variance explained by X,gives the proportion of Y variance that is uniquely explainedby X via M.

R2ind ¼ r2Y M :Xð Þr2MX ð8ÞAs a product of two squared numbers, R2ind will never benegative. And because r2MX ¼ b2MX , and because it followsfrom exchanging X and M in eq. 7 that r2Y M :Xð Þ ¼1� r2MX� �

b2YM :X , eq. 8 can be rewritten as follows.

R2ind ¼ 1� r2MX� �

b2YM :X b2MX ¼ 1� r2MX

� �b2ind ð9Þ

Comparing eqs. 7 and 9 reveals that now the direct andindirect effect are treated in a completely symmetricalway. For both effects, the unique proportion of varianceexplained is computed by multiplying the squared betawith 1� r2MX

� �, so identical betas for the direct and

indirect effect lead to identical R squared measures ofvariance explained.

As in the approach based on squared betas, there is ajoint part of variance explained, which now equals2bdirbind þ r2MX b2dir þ b2ind

� �. In comparison to the squared

beta approach, this joint part is more positive if the twoeffects are of the same sign, and less negative if they are ofopposite signs (unless rMX = 0, in which case it makes nodifference). As before, we can use the unique or thehierarchical approach for assigning the three components

of variance explained to the direct and the indirect effect,with the same advantages and disadvantages as discussedbefore. The difference is that in the adjusted Fairchild et al.(2009) approach, the unique contribution of an effect iscorrected for the other effect as it is in the present data,whereas in the approach based on squared betas the uniquecontribution of each effect is computed for the morehypothetical situation if the other effect would be zero.

After having derived this measure, I found out thatalthough not described in Fairchild et al. (2009), aclosely related measure was mentioned in MacKinnon(2008, p. 84, equation 4.6) as one of three measuresrequiring more development. The only difference is thatinstead of a semipartial correlation with X partialled outfrom M only, as in the present article, MacKinnon (2008)used a partial correlation of Y and M with X partialled outfrom both Y and M, leading to R2ind ¼ r2YM :X r2MX . The effecton interpretation is that we are addressing the samevariance in the two measures, but as a proportion of adifferent total in each, namely all Y variance (semipartial)versus Y variance not shared by X (partial). Both measuresare equally valid as long as we are clear about from whichtotal we take a proportion, but to my opinion, using thesemipartial is preferable over the partial, because it iseasier to understand, measures the direct and indirecteffect as proportions of the same total instead of differenttotals, and is completely symmetric in its treatment of thedirect and indirect effect.

Discussion and conclusions

Limitations

Three possible limitations of the present study shouldbe discussed. First, one might wonder whether therestriction of discussing R squared effect sizes only inrelation to standardized effects (betas instead of b’s)does limit the generality of the results. It does not,because R squared effect size measures are standardizedthemselves, completely based on correlations, which arecovariances between standardized variables. We go fromstandardized to unstandardized effects by multiplying thebetas for the total, direct, and indirect effect all with thesame constant (SD(X) / SD(Y)), and apart from that,nothing changes.

Second, nothing was assumed or said about thedistributions of X, M, and Y. A minor point here is thatdifferences between the distributions of the three variables(e.g. when X is strongly skewed, but Y is not) may limit themaximum size of correlations, and therefore of R squaredeffect size measures. Apart from this minor point, it can beargued that for the purpose of the present article,

Behav Res (2012) 44:213–221 219

distributional assumptions were not necessary, because noattempt was made to estimate stability and bias of theeffect size measures.

This absence of simulation studies in order establishstability and bias of the R2ind measure of eq. (8) is the thirdand most serious limitation of the present study. At present,the only information I can give comes from MacKinnon(2008, p. 84), who for his closely related measure (identicalexcept for the use of partial instead of semipartialcorrelations) reported that in an unpublished masters thesis(Taborga 2000) minimal bias, even in relatively smallsamples, was found. Whether this generalizes to the presentmeasure is a matter for future research.

Conclusion and recommendations

The most important conclusion of the present study is thatin terms of variance explained there is strong overlapbetween the direct and indirect effect. In order to handleand quantify this overlap, a method has been presented todecompose variance explained into three parts (uniquedirect, unique indirect, and joint part). Because of thisstrong overlap, dividing variance explained over the directand indirect effect is only possible if we make some choiceabout what to do with the joint part.

In a way, there is nothing new here. Handling suchoverlap is a routine matter in ANOVA for unbalanceddesigns and regression analysis with correlated predic-tors, and the unique and hierarchical approaches sug-gested in the present article are closely parallel to themost commonly used ways of handling overlap inregression and ANOVA. However, due to the intercon-nectedness of the direct and and indirect effect, theamount of overlap is much larger than what we usuallyencounter in regression and ANOVA, so differentchoices of how to handle overlap may lead to radicallydifferent effect sizes (as was illustrated by the examplesin the present study). Unfortunately, choices of how tohandle such overlap are always arbitrary to some extent.

Is it possible to give some useful advice to the appliedresearcher? Due to the absence of knowledge of possiblebias of the R2ind effect size measure and the impossibilityto eliminate all arbitrariness from the decision aboutwhat to do with the overlap part of variance explained,very specific guidelines are impossible. However, somegeneral recommendations for using R squared effect sizemeasures can be given.

1. Generally, the approach based on squared semipartialcorrelations should be preferred above the approach basedon squared betas. In the present article, squared betaswere useful for explaining the interconnectedness of thedirect and indirect effect, but as measures of unique

variance explained they are too hypothetical for almostall research situations in the social sciences.4 Measuresbased on squared semipartial correlations are much moredescriptive of the actual data.

2. If there are good reasons to indicate one effect asprimary, or if the two effects are of opposite sign, use thehierarchical approach. In many situations, it is verynatural to ask for the unique contribution of the primaryeffect, and then for the additional contribution of thesecondary effect. If we have good reasons to choose thedirect effect as the primary one, the original Fairchild etal. approach still is the thing to do.

3. If there are no good reasons for a primary versussecondary effect distinction, use the unique approach.The price to be paid is that a lot of overlap varianceremains unexplained, but this is not too different fromwhat we routinely accept when doing ANOVA ormultiple regression analysis.

4. Whatever we do, in terms of R2 and varianceexplained, there will always be substantial overlapbetween our effects and some arbitrariness in ourhandling of this overlap. If we accept this, we can makeour decisions as described in the three points above.Alternatively, we might conclude that R2 measures are notthe best possible way to describe effect size in mediation,and consider other effect size measures, that possiblysuffer less from overlap between effects. We mayreconsider proportion mediated as a measure of effectsize, or we could simply use bind (eq. 1c, discussed as thecompletely standardized indirect effect by Preacher andKelley (2011), or we could try one of the other measuresdiscussed by these authors. At present, I see no measurewhich is satisfactory under all circumstances, so thequest for the perfect measure of effect size in mediationshould go on.

4 In principle, squared betas can be useful if we want to predict howmuch variance would be explained by an effect if the other effect wereblocked somehow. It is possible to imagine research situations inwhich this question has some use. For example, some experimentalmanipulation (X) may have both a direct effect on general mood (Y)and an indirect effect by inducing fear (M), which in turn influencesgeneral mood. Now if we could modify the experimental manipulationin a way that eliminates its effect on fear, without changing its directeffect on mood in any way, the squared beta of the direct effect in ouroriginal experiment would predict the proportion of varianceexplained in that new situation. However, even in this artificialsituation, computing variance explained for a new experiment with themodified manipulation would be strongly preferable over assuming itsvalue via squared betas, because it is an empirical question whetherthe direct and indirect effect of the modified manipulation will behaveas expected by the investigator.

220 Behav Res (2012) 44:213–221

Open Access This article is distributed under the terms of theCreative Commons Attribution Noncommercial License which per-mits any noncommercial use, distribution, and reproduction in anymedium, provided the original author(s) and source are credited.

References

Alwin, D. F., & Hauser, R. M. (1975). The decomposition of effects inpath analysis. American Sociological Review, 40, 37–47.

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Appliedmultiple regression/correlation analysis for the behavioral sciences(3rd ed.). Mahwah, NJ: Erlbaum.

Fairchild, A. J., MacKinnon, D. P., Taborga, M. P., & Taylor, A. B.(2009). R2 effect-size measures for mediation analysis. BehaviorResearch Methods, 41, 486–498.

MacKinnon, D. P. (2008). Introduction to statistical mediationanalysis. New York: Erlbaum.

MacKinnon, D. P., & Dwyer, J. H. (1993). Estimating mediated effectsin prevention studies. Evaluation Review, 17, 144–158.

MacKinnon, D. P., Warsi, G., & Dwyer, J. H. (1995). A simulationstudy of mediated effect measures. Multivariate BehavioralResearch, 30, 41–62.

Mood, A. M. (1969). Macro-analysis of the American educationalsystem. Operations Research, 17, 770–784.

Preacher, K. J., & Kelley, K. (2011). Effect size measures formediation models: Quantitative strategies for communicatingindirect effects. Psychological Methods, 16, 93–115.

Seibold, D. R., & McPhee, R. D. (1979). Commonality analysis: Amethod for decomposing explained variance in multipleregression analyses. Human Communication Research, 5,355–365.

Stevens, J. P. (2007). Intermediate statistics. A modern approach (3rded.). New York: Erlbaum.

Taborga, M. P. (2000). Effect size in mediation models. Arizona StateUniversity, Tempe: Unpublished master’s thesis.

Behav Res (2012) 44:213–221 221

R squared effect-size measures and overlap between direct and indirect effect in mediation analysisAbstractR squared measures for the direct and indirect effectThe total, the direct, and the indirect effectEffect size

ProblemsA counterintuitive exampleAsymmetry of direct and indirect effect

Explanation and solutionsDirect and indirect effect are not independentTwo ways of dividing variance over direct and indirect effectNegative variance explainedComparison with the Fairchild et al. approachModifying the Fairchild et al. approach

Discussion and conclusionsLimitations

Conclusion and recommendationsReferences