Managers’ selection: Survivors always look better, whether they are skilled or not

7/29/2019 Managers selection: Survivors always look better, whether they are skilled or not

1/13

1

Managers selection: Survivors always look better,

whether they are skilled or notBy Dominic Clermont, ASA, MBA, CFA - ClermontAlpha.com

February 7, 2012

Survivorship bias is present and part of our due diligence process. Whether a

managers performance database is free of survivorship bias or not, thetypical use of the database by investors and consultants induce such effect and

may lead them to improper conclusions and decisions. The impact of

survivorship bias is much greater for low-skilled managers making it virtually

impossible to distinguish between skilled and non-skilledsurviving managers

based solely on performance. Survivorship bias also has a much greater impact

on high risk investment strategies, something to keep in mind when considering

highly concentrated portfolios. Finally, survivorship bias may induce positivecorrelation between value added and active risk, even when there is no

correlation (i.e. for non-skilled managers).

While I was heading a risk-controlled active group at a leading bank, one of my competitors,

working for a subsidiary of a leading global manager, developed a competing product which was

sold as a quantitative strategy with a judgmental overlay. Their marketing (which included their

parent clout) was very successful at attracting clients.

Unfortunately for their clients, the performance was very negative leading the fund to have the

worst performance (last percentile ranking in a leading managers survey) over 3, 4 and 5 years.All clients exited the underperforming strategy and the investment team was dismantled. I tried

to obtain its historical performance from a survey prepared by one of the largest global

investment consulting firm who pride themselves of having a database of managers with no

survivorship bias; unfortunately they did not have any data on this defunct fund anymore.


2/13

2

Survivorship bias is responsible for a significant positive drag effect in managers performance

survey.

Another investment consulting firm is recommending a more active management approach at a

much higher risk based on the following analysis:

Exhibit 1: Active Return vs Active Risk for Global Mandates

The analysis refers to global mandates and a global benchmark. Their observations and

recommendations are:

There is significant value added from global mandates. There is a strong correlation between active return and active risk. Since return increases with risk, we should not be afraid of concentrated/more risky

portfolios.

The slope of 0.63 is clearly significant and means that for every increase of 1% in risk for the

average manager, value added increases by 0.63%. In other words, the information ratio of the

average manager is a spectacular 0.63. From the chart in exhibit 1, we can also observe many

managers with an IR of around 1.

While global markets generally offer a better risk-return profile due to their lower efficiency, the

above observations are suspicious.

The effect of survivorship bias is much more important than investors think!

Investors and consultants do not fully appreciate the difficulty of differentiating skilled and non-

skilled managers. The managers that investors and consultants meet are those who have

survived till today. The impact of surviving (and/or screening the underperforming) can easily

be illustrated in a simple scenario where we would screen out managers with negative active

(excess) returns. If we simulate a large numbers of unskilled managers with expected active

returns of 0%, and active risk of 1%, the distribution of active returns would look like this:


3/13

3

Exhibit 2: Return distribution with Expected Active Return of 0% (no skill) & Active Risk of 1%

If we screen out the underperforming managers, we obtained the following truncateddistribution:

Exhibit 3: Truncated Return distribution with Expected Active Return of 0% (no skill) & Active Risk of 1% withnegative returns eliminated

This distribution would be observable for managers with no skills (expected returns = 0%) and

one percent active risk. By eliminating those non-skilled managers who experienced negativereturns, we are left with the lucky ones. The lucky non-skilled managers would have a median

outperformance of 0.67% and an average outperformance of 0.80%. For a typical level of active

risk, we could redo the above simulation at say 4% active risk and we would obtain the same

numbers as above multiplied by 4, i.e. a median excess return of 2.67% and an average return of

3.20%.


4/13

4

In fact, for no-skilled managers with positive return by chance, their

median Information Ratio over one year is 0.67 and their average

IR is expected to be 0.8a performance quite exceptional!

Every investor and consultant would love to know for each of their managers their true Ex-ante

IR, i.e. the IR we can expect from that manager if we invest with him over a long period oftime. Unfortunately, this measure is unknown. We can only measure a managers Ex-post IR

over a period of historical time which is more or less long depending on the manager. Even if

the Ex-ante IR would be known, it is not static as the manager changes and the economy

changes. We can only hope that a manager with some positive investment skill will manage to

maintain that skill and deliver some consistent IR. In our simulation, we assume that we know

the Ex-ante IR (and that it is fixed over time) and look at the impact of screenings and managers

disappearance on the Ex-post/observed IR.

10-year simulation

In practice, managers do not get screened-out after only one year of negative performance. We

generally ask for three years of returns. For large established firms, we may tolerate some small

negative performance (which is why my former competitor lasted for more than 5 years with bad

performance). I did two sets of simulation for two level of ex-ante IR:

IR of 0 (non-skilled managers) and IR of 0.3 (skilled managers) Active risk from 1% to 14% Threshold before firing the manager: 3-year rolling outperformance of below -2% Simulated returns are log-returns in order not to generate any negative bias in

compounded returns.


5/13

5

10-year simulation - IR of 0 (non-skilled managers)

Exhibit 4: 10-year simulation for IR of 0 (non-skilled managers)

As expected, the average simulated IR is 0. The trend line has a slope of 0 meaning no value

added for the average manager, independent of the active risk level. The average expected value

added is 0 as expected. All managers above the zero line were lucky managers with positive

alpha and positive ex-post IR, while those below the zero line were unlucky managers with

negative alpha and negative ex-post IR. Some were very lucky with high alpha and high ex-post

IR.

The volatility of the annualized return should be:

This is why we observe a dispersion of performance around the expected value added of 0%

which increases with risk. It is important to note that, independent of how large the ex-post/observed value added is, all these simulated managers have the same ex-ante expected value

added of 0% - i.e. thay all have no skill.

If we screen out any manager with a negative 3-year rolling outperformance below -2%, we end

up keeping all lucky managers having all of their overlapping 3-year return above -2% during the

10 year of the simulation. We obtain the following chart. If this chart looks familiar, it is


6/13


7/13

7

10-year simulation - IR of 0.3 (skilled managers)

We redo the same exercise, this time assuming an IR of 0.3.

Exhibit 6: 10-year simulation for IR of 0.30 (skilled managers)

For all simulated managers, the slope of the trend line is close to 0.3 (t-stat above 13) showing a

clear investment skill for this group of managers. Note that while all these simulated managers

have the same investment skill, thus the same expected value added per unit of active risk, many

of them are unlucky, even after 10 years, and show negative value added over the whole period.

No clients would tolerate their skilled managers to underperform for 10 years. In fact, everyone

would wrongly classify them as non-skilled.

If we screen out any unlucky skilled manager with a negative 3-year rolling outperformance

below -2%, we end up keeping all managers having all of their overlapping 3-year value added

above -2% during the 10 year of the simulation. We obtain the following chart:


8/13

8

Exhibit 7: 10-year simulation for IR of 0.30 (skilled managers) keeping only the survivors

As for the non-skilled managers, the skilled survivors see their average IR improve but the

impact is less significant as the average IR increases from an expected 0.3 to an observed 0.61.

For no-skilled managers, IR increases from an expected 0.0 to an observed 0.51.


9/13

9

10-year simulationCombining both simulations: Ex-ante IR of 0 and 0.3

If we combine both groups (after screening), we obtain a graph showing how difficult it is to

distinguish between a skilled manager and a non-skilled one.

Exhibit 8: 10-year simulation with both skilled and non-skilled managers - keeping only the survivors

The screening process has a much greater positive impact on the non-skilled managers IR.While the no-skilled managers have a higher death rate compared to the skilled ones, the

conclusion of this exercise is that we cant base our decision on performance, as the no-

skilled manager looks almost as good as the skilled one.

Survival Rate

The above analysis would not be complete without talking about survival rate. When we apply a

realistic screening process as in the simulation above, we observe, as expected, a higher survival

rate for skilled managers vs lower-skilled ones. We also observe a negative relationship between

survival rate and active risk. This means that the death rate increases with risk whether a

manager is skilled or notsomething to keep in mind when contemplating highly concentrated

strategies.


10/13

10

Exhibit 9: Survival rate after 10-year simulation keeping managers with rolling 3-year return above -2%

For the skilled managers with an ex-ante IR of 0.3, one third of the low risk managers have

disappeared after 10 years and over 70% of the skilled high risk managers are gone. For the non-

skilled managers with an ex-ante IR of 0.0, 43% of the low risk managers have been lucky

enough to still be there after 10 years, while this survival rate drops to slightly less than 10% forthe higher risk onesan incredible performance for managers with expected value added of 0.

Types of Survivorship Bias

While the literature document two types of survivorship bias, there are really three:

1. Survivorship biasPast return of managers who ceased reporting

This bias occurs when a managers database exclude all or part of the returns of dead,

merged or otherwise dissolved funds. Since dead funds generally had poor returns before

ceasing to report, excluding their history from the database cause the reported

performance in those years to overstate the true average return available to investors atthe time.

This effect is relatively well researched and documented in the literature. Damato [1997]

report a mutual fund performance by Lipper Analytics. In 1986, 568 diversified U.S.

stock funds had delivered an average 1986 return of 13.39%. 10 years later, Lipper


11/13

11

provided a different picture of the same 1986 performance as there were only 434 funds

remaining with an average return of14.65%.

Some database providers now market their product as free of survivorship bias. My own

experience, as asking for my defunct competitors historical performance showed, is that

this assertion is not always valid. Furthermore, typical usage by investors and

consultants do create subsamples that do have survivorship bias. Investors will typically

focus on managers with positive track records. Using my simple rolling 3-year screen

described previously, I obtained the following interesting statistics:

ACTIVE RISK

4% 8%

IR=-0.50 1.78% 4.81%

IR=-0.20 1.11% 3.50%

IR=0.00 0.73% 2.47%

IR=0.30 0.33% 1.31%

IR=0.50 0.19% 0.80%

Exhibit 10: Year 1 performance improvement when applying therolling 3-year screen (deleting funds with performance below -2%

at the end of year 3)

The interpretation of this table is as follow: Skilled managers with an IR of 0.3 have

expected value added of 1.20% (IR of 0.30 times 4% active risk). Reported value added

for year 1 increases by approximately 0.33% (from 1.20% to 1.53%) at the end of year 3

when we screen out underperforming managers.

Reported performance increases significantly when eliminating underperformers

and looking only at survivors. And that increase is higher for lower skilled

managers and for managers taking higher active risk.

Database which keeps the historical track record of defunct funds reduce, if not eliminate,

this impact of survivorship bias in reporting historical average performance by all

managers. As we will see in a moment, it does not eliminate all survivorship bias effects.

2. Backfill bias or Instant History Bias

Backfill bias occurs when a database supplier backfill a new funds performance when

added to its database. This leads to overstatement of the backfill period average

managers performance as funds would generally be added after good starting value

added.


12/13

12

If only the managers with positive first-year value added were accepted to report to the

database, these one-year positive new-comers would include:

- 50% of the non-skilled managers (IR=0) with an observed average IR of 0.80- 62% of the skilled managers (IR=0.3) with an observed average IR of 0.92Some firms try to eliminate the backfill bias effect by eliminating the historicalperformance of newcomers. This is probably wrong as the historical performance

belongs to these firms. When reporting the average return for that period, backfilled

data should be excluded to take into account that new managers with negative first year

performance most likely did not report.

3. Survivorship biasscreening the database

Even if the managers database is free of survivorship bias, its use often involves

significant survivorship effect. When considering investment managers, investors and

consultants screen for managers with a certain positive historical performance over 3, 5

or 10 years. If we screen the database for all managers with a 10 year history, defunctfunds are evidently eliminated, inducing a survivorship bias effect something that was

documented by Cathcart [1994]. Such screening process has exactly the same effect as

the one described in this paper. The fact that database suppliers keep defunct funds in the

database for the period they were managing money does not cancel the survivorship bias

faced by hiring investors and their consultant in their use of such database. To be totally

clear of such survivorship bias effect, the database would need to include the

performance of funds even after they have stopped reporting something not possible for

dead funds.

Conclusion

Survivorship bias in managers database may come from the database maintenance which does

not keep historical information of funds that ceased reporting. Even if database providers keep

dead funds historical performance, survivorship bias is still present (and neglected) in the use of

the database. Its effect is to induce and/or increase the correlation between value added and

active risk even for non-skilled managers (IR of 0). Investors and consultants should be

reminded that the IR of survivors often overstates their true ability.

The survivorship bias effect has much greater impact on low-skilled managers making it

virtually impossible to distinguish between skilled and non-skilled managers based only onhistorical performance.

The positive correlation, observed between value added and active risk, leads some investors and

consultant to seek more concentrated portfolios. The survivorship bias effect increases with

active risk and, again, may induce false correlation between value added and active risk for non-

skilled managers. Investment in more concentrated portfolios should be based on true measure

of an investors investment skill. IR is not a measure of skill and is only weakly correlated to


13/13

13

skill in the long run. Investors should keep in mind that it takes 16 years to be 95% confident

that a manager with an IR of 0.5 delivered value added from skill, not luck. We could still be

wrong 5% of the time! With managers with an IR of 0.30, it takes more than 40 years. And

these observed IR are those of survivors thus they generally overstate their true IR.

Information Coefficient (IC) is generally seen as a more proper measure of an investment skill

but requires a lot more information and analysis by investors and consultants.

Mutual Fund Survivorship, Mark Carhart, doctoral dissertation, 1994

Dead-and-Buried Funds Would Alter 'Averages' - If Included, By KAREN DAMATO, Wall

Street Journal, April 4, 1997

Managers’ selection: Survivors always look better, whether they are skilled or not

Documents

Transcript of Managers’ selection: Survivors always look better, whether they are skilled or not