Measuring the Immeasurable: Market Lucidity Index

8/9/2019 Measuring the Immeasurable: Market Lucidity Index

1/23

Alexandre Pestov Eugene Bragin May 2010

Measuring the Immeasurable: Market Lucidity Index Page 0 of 51

Measuring the Immeasurable: Market Lucidity Index

MAY 2010

Alexandre Pestov Eugene Bragin


2/23



ABSTRACT

This research was conducted to develop the Market Lucidity Index, an index that offers a relatively quantitative

assessment of the markets inclination with respect to interpreting news bearing significant impact on equity

markets. This seemingly unachievable goal was accomplished through establishing a rigorous quantitative

framework for evaluating market expected reaction relative to actual.

The Market Lucidity Index proposed in this paper carries no forecasting value. It is instead a reflection of the

dominant market sentiment with respect to the interpretation of significant economic events. Unlike the P/E ratio

or purely technical indicators like Bollinger Bands and RSI, the index detailed in this paper acts as a litmus test of

the behavioural discrepancy between the markets expected reaction and its actual. In conjunction with other

indicators, however, it can be used a tool to help identify possible issues with the market (e.g. herding behaviour,

severe optimism and pessimism periods) from the behavioural perspective.

By evaluating the research results, we concluded that in 2009 the equity markets exhibited extreme levels of

optimism, signalling possible disconnect of the market participants reaction from the underlying economic

developments. The index suggests that the fine s ignals, or green shoots of a typical recovery, did not exist to

support the rally the way it progressed. Instead, the market went up on sheer optimism and hope, fuelled by

unprecedented liquidity measures developed by governments and central banks around the world. A stock market

action based on emotions alone possesses tremendous opportunities and danger to the investors, depending on

the preceding market action. As the markets rallied on overwhelming optimism in 2009-2010, it is our view that

retail investors should maintain minimal exposure to equities until both markets behaviour and value come in sync

with the long-term norms.


3/23



TABLE OF CONTENTS

Abstract ..................................................................................................................................................................................................... 1

1 Introduction .......................................................................................................................................................................................... 3

2. About The Project ............................................................................................................................................................................. 6

3. Methodology ........................................................................................................................................................................................ 7

3.1 Inputs .............................................................................................................................................................................................. 7

3.2 Assumptions and General Approach ..................................................................................................................................... 8

4. Analysis................................................................................................................................................................................................ 10

4.1 Defining Relevant Economic Indicators ............................................................................................................................... 10

4.2 Filters on the S&P 500 Index.................................................................................................................................................. 10

4.3 Newscast Content .................................................................................................................................................................... 11

4.4 Step 1: Calculating Component Changes............................................................................................................................ 11

4.5 Step 2: Calculating Index Sensitivity ..................................................................................................................................... 12

4.6 Step 3: Measuring Bias Strength ............................................................................................................................................ 12

4.7 Step 4: Finalizing Individual Components ............................................................................................................................ 13

4.8 Step 5: Finalizing The Index .................................................................................................................................................... 17

5. Parting Words ................................................................................................................................................................................... 19

Acknowledgements .............................................................................................................................................................................. 22


4/23



1 INTRODUCTION

An optimist believes we live in the best of all possible worlds.

A pessimist fears this is true.

Murphy's Law

The financial and economic areas of human activities, now fully reliant on instant and global communication

exchanges, are amongst the most striking phenomena distinguishing the contemporary society from the previous

eras. They started as frequent human interactions, but they now moved almost entirely into the virtual world.

Myriads of digits symbolizing wealth and capital shift up and down, reflecting ever-changing expectations of

individuals, companies and entire countries that collectively form what we know as the market. These constant

movements occupy a worthy place in our daily lives. Despite their digital nature, they remain tremendously

important and strangely captivating to human humanssimilarly to the powerful natural forces that cause

hurricanes, earthquakes and other significant events.

As it is the case with dealing with significant natural forces, sophisticated observers have developed gauges to

measure movements of the markets known as indexes. Market indexes are many and show different facets of a

specific asset class in a specific space. Typically, each respective index employs a number of unique methodologies

for calculating index values. They can be simple or sophisticated. However, they all are used to consolidate and

simplify complex realities in order to facilitate their understanding.

Indexes are part of human-made notional arrangements. They combine collective behaviour exhibited by myriad of

market participants. Gauges invented to measure natural forces provide necessary degree of certainty for making

decision. By looking at thermometer we can say whether it is cold or hot, which will help to determine the optimal

course of action, choosing appropriate cloth to wear in this case. Readings of a market index generally dont carry

the same suggestive powers. Nor they directly impact most of those who actively monitor them. As a rule, most

index watchers dont get richer when index rises or poorer when it falls. Rare exceptions to this rule includesuccessful day traders and sophisticated HFT algos. Yet, there is no shortage of those who obsessively monitor the

various indexes and not just in the worlds financial hubs. Presently, even in the most remote towns of Siberia,

where residents dont have a slightest idea of who or what Dow Jones is and why it should remain above the

psychologically important level of 10,000, TV broadcast take it as their sworn duty to deliver intra-day updates on

DOWs movements. It is important for the markets, then, to establish secondary indicators based on the

underlying mood or condition changes, which help to measure the strength of prevailing forces and aim to

establish future direction. Some examples of it include TED spread and VIX index.

Since the beginning of the financial crisis in 2007 the market has gone through the phases of remarkable volatility.

The massive global sell-off of 2007-2009 was followed by a vigorous comeback staged by the global markets in

2009. And while the events of 2008 could be expected due to the deteriorating fundamentals and liquidity freeze,

the briskness of the market reversal was quite surprising. What was even more surprising is the swiftness by which

market gains outpaced improvements, or lack of thereof, in underlying economic fundamentals. Starting with the

green shoots that Ben Bernanke observed in mid-March of 2009, the recovery somehow has never felt

genuine. Even as the illusionary green shoots spread roots in minds of market participants, the situation

continued to feel surreal, even as the long-awaited market recovery ensued. Every green day on the market

seemingly went against all odds, as if the market was a mirror copy, a complete opposite, of what should have

occurred. And while it felt like a make-believe game at the beginning, with time it exceeded even wildest

expectations. The market seemed to be bullish, no matter what (from The Daily Bell).


5/23



It's BULLISH No Matter What! ...

The price of oil is rising - BULLISH! More profits for the energy companies, and more investments in "clean

energy."

Most of the new jobs created in March were part-time or temporary - BULLISH! Since the economy has turnedthe corner full-time job offers are practically a sure thing.

But didn't wages go down too? - BULLISH! Revenues - Costs = Profits!

41 states have revenue shortfalls - BULLISH! Various states have always complained about shortfalls. It's another

sign that things are getting back to normal.

8 million people are still unemployed - BULLISH! That's 8 million spenders, not savers.

Interest rates are rising - BULLISH! Yet another sign that the economy is getting stronger.

Stocks may be going up but on very low volume - BULLISH! That means the "dumb money" hasn't even bought

into this rally yet.

People have a lot of concerns and uncertainty about the future - BULLISH! Not until the "wall of worry" ends will

this party be over.

So much new liquidity will cause inflation - BULLISH! Stocks are one of the best hedges against inflation.

The wars in Iraq and Afghanistan are bankrupting us - BULLISH! Don't get mad, get even. Debit the Treasury

and Credit the defense companies.

Inflation in China is picking up - BULLISH! That should dampen any bubbles that some people worry about.

Gold is going up in price - BULLISH! This is a broad-based rally.

Wait, maybe gold is going down - BULLISH! That means economic fears are dissipating.

Actually the gold price seems to be consolidating and moving sideways - BULLISH! A sell off or rally would mean

things are overheating.

Iran seems determined to develop it's nuclear program - BULLISH! More nuclear power plants means less

demand on oil which means lower energy costs which means more profits.

Israel may be forced to handle Iran themselves militarily - BULLISH! That will kick-start the construction industry

when we rebuild both sides.

The Health Insurance Reform bill is an abomination - BULLISH! If insurance premiums rise there will be

subsidies; if doctors check out they'll be replaced with cheap foreign ones; if care is rationed then costs will be

controlled and profits ensured.

And now the student loan programs are nationalized - BULLISH! Good riddance for the banks. Now the

government can garnish wages and lower the deficit.

The markets are being purposely manipulated with government money - BULLISH! What's not to like? That

means the market ain't going down no matter what.


6/23



Big Media is spewing propaganda about the economy - BULLISH! Perception is reality. People only know what

they're taught. Advertising works.

Greece may default - BULLISH! Greek bond holders will make up their loses in the stock market.

Japan is a bug in search of a windshield - BULLISH! Just imagine how much more deficit spending we need to doto beat them.

The Euro is getting weaker - BULLISH! King dollar is back.

Source:http://www.thedailybell.com/949/Happy-Days-Are-Here-Again.html

We therefore found it useful to develop a framework for validating market lucidity - extreme bullishness or

bearishness - in quantifiable terms. The end objective of our project was to examine the market and identify any

possible interference that must be considered by market participants. We established our hypothesis to test as:

Actual market movements can be evaluated in their correlation to the

expected market direction to identify positive or negative bias dominating

the markets at a given point in time.
http://www.thedailybell.com/949/Happy-Days-Are-Here-Again.htmlhttp://www.thedailybell.com/949/Happy-Days-Are-Here-Again.htmlhttp://www.thedailybell.com/949/Happy-Days-Are-Here-Again.htmlhttp://www.thedailybell.com/949/Happy-Days-Are-Here-Again.html


7/23



2. ABOUT THE PROJECT

To test the hypothesis it was imperative to restrain our natural scepticism towards stock market efficiency. As the

first step, we had to eliminate our personal doubts about the adequacy ofmarkets behaviour and questions about

rationality of their existence. Instead, the focus of our research was on finding a rational basis that guides market

participants.

Financial news and data releases form the basis of knowledge that influences decisions of market participants.

Again, it is necessary to define stringent criteria to reduce accidental information perception, excessive amount of

irrelevant news events that form the white noise, which is a direct result ofsubjective news interpretations and

occasionally direct market manipulations. In the end, speculators in the market do not engage in a profound socio-

economic analysis. Instead they rush to the surface of the conventional market-moving factors, trying to guess the

intentions of other market players who, by and large, are driven by the same instincts and considerations.

Therefore, it seems quite acceptable and warranted to consider news information releases that carry market-

influencing powers - as a key source of pulses for market fluctuations. It must be mentioned that these news

releases are sketchy at the best: : They exist only a day or two at most and are soon deleted, discarded, and

replaced by the next urgent headline.

From this reasoning was born the basic idea behind this paper: Measure unpredictable market movements by

evaluating equally unpredictable news releases. At first glance, the idea of using news to measure the market

sounds as highly complex. However, the task of establishing correlations between news and the market is

simplified by these four essential assumptions:

Macroeconomic or major financial events, such as earnings releases, unemployment and GDP changes, areexpected to have the greatest consistent influence over the behaviour of speculators;

Generally, news releases can be defined by a set of three critical numbers: values before and after, andexpected value (consensus);

Index performance is well documented, and so are the news releases, which allows for evaluating possiblecorrelations;

Finally, the original conviction in rational perception of news releases allows restricting the field of studyto the two data series: market performance and corresponding consolidated index of news releases.

And so we defined the objective for this paper: To construct an index that offers a relatively independent and

unbiased assessment of the market lucidity.


8/23



3. METHODOLOGY

3.1 INPUTS

The scope of this paper is confined to examining the lucidity of US stock markets participants. Although numerousstock indexes can be used to track market activities, we selected S&P 500 as a sufficiently representative and the

most appropriate indicator of the broader US stock market. Additionally, the S&P 500 index contains a complete

array of the most significant companies listed on the U.S. exchanges, and consolidated weighted earnings release

evaluation for them can be included as a complimentary component to enhance the Market Lucidity Index .

From the news analysis perspective, we limited the extent of our research to the news range that can be

quantified. News feeds are delivered to the worldwide consumer marketplace continuously, streaming down from

countless TV screens, web portals, newspapers and other media outlets. Collectively, news is one of the most

persuasive influences on the public opinion, with the power to reinforce or significantly challenge peoples beliefs.

Even the smallest changes to the emotional tone of newscasts have the potential to affect behaviour of individual

market participants. If the daily news flow is sufficiently voluminous and convincing, these individuals begin voting

with their feet and wallets collectively. In the most meaningful phase, these market participants begin to network

among themselves, creating a market culture that becomes self-determinant and mutually reinforcing, materially

impacting the broader market. For the purposes of this research, however, we assumed that myriads of newscasts

create generally net-neutral white noise that should be filtered out from, rather than included in, our analysis.

Furthermore, the sheer volume of newscasts and unfeasibility to accurately quantify their individual market impact

dictates elimination of this white noise from the news impact calculation. As a result, we focused our attention

on the generally accepted economic indicators that have known, and thus possibly quantifiable, effects on the

market performance. Table 1 lists the indicators evaluated for the purposes of this research.

Table 1: Key Economic Indicators

Economic Indicator

Auto and Truck Sales

Business Inventories

Construction Spending

Consumer Confidence

Consumer Credit

CPI : Consumer Price Index

Durable Goods Orders

Employment Cost Index

Existing Home Sales

Factory Orders

GDP : Gross Domestic Product

Housing Starts and Building Permits

Industrial Production

Initial Claims

International Trade

Leading Indicators


9/23



M2

NAPM: National Association of PurchasingManagersNew Home Sales

Personal Income and Consumption

PPI: Producer Price Index

Productivity and Costs

Regional Manufacturing Surveys

Retail Sales

The Employment Report

Treasury Budget

Weekly Chain Store Sales

Wholesale Trade

Additionally, we included a supplementary model that was adjusted for an accumulative S&P 500 companies

earnings impact as described later in this document.

News releases for each of the economic indicators were evaluated using the three key components:

Actual value of the component as presented in the newscast; Previous value of the same component; Estimates or expected value of the component (market consensus).

These three values were linked using the newscast dates to the S&P 500 index performance to create the four

pivot points of our analysis. Since the objective of our research included measurement of the long-term market

fluctuations, the time range of our study was limited only by the availability of reliable information and data series.

This analysis is made on the basis of the data collected for the range of January 1, 1998 to May 15, 2010.

3.2 ASSUMPTIONS AND GENERAL APPROACH

As noted in the introduction section of this document, we assumed that the market is driven by a Brownian

motion of pseudo-rational actions. These actions are numerous, sporadic and frequent. Although collectively they

form the broader market shifts, individually they exist only on a very short time line, and small sporadic

movements of individual market participants effectively cancel each other out, having net-zero effect on a broader

market. With this in mind, as well as the assumption of that the main driving force behind the market shifts is the

economic health at the moment in time as perceived by the market participants, we established that an unbiased

market will move up in reaction to positive economic news or shift down in response to negative. The schedule

for each news release tied to the corresponding market movements allowed isolating individual components inorder to assess the market impact of each newscast.

Certainly, limiting market impact evaluation to a set of scheduled economic indicators and earning releases does

not accurately reflect all sentiments prevailing in the market. There are plenty of other factors that affect the

minds of speculators. Since our objective was to create a relative evaluation of market lucidity, the presence of

other news can be combined and encapsulated into a notion of market bias. Then a background informational

(or noise) will be expressed as a disagreement with the expected rational market behaviour, and thus allow

identifying the nature of the bias. For example, if the market moves up on negative news, we can conclude that a


10/23



positive bias exists within the system. Similarly, if the market falls on positive news, we determine that the market

bias is negative. On a long enough timeline this approach will allow to indirectly integrate the analysis of the sum of

all expectations to determine the prevailing mood of the market and identify lapses in market ability to interpret

news.

To simplify the analysis, in situations when the expected market reactions matches the actual, the mismatch in

reaction sensitivity (e.g. on a massively positive news the market reaction is positive, although fairly muted) isignored. For example, the market reaction to an unemployment rise of 0.5% was as follows:

6 June 2008: -1.8% January 9, 2009: - 4% March 6, 2009: - 2.5% June 5, 2009: - 0.004%

Although in all cases, rising unemployment resulted in falling markets, the reaction was much stronger on January

9, 2009 than in June 5, 2009, when it showed only symbolic move down. Such sensitivity analysis requires more

thorough investigation and modeling, and it will be reserved for the future research. However, for the purposes of

this paper, only correlations of news and the S&P 500 index changes direction were considered.

To smooth out fluctuations and reduce the impact of outliers, moving averages were used where applicable. To

factor in actual component changes as well as market expectation, a combination of value change (current vs.

previous value) and missed/exceeded expectations (current vs. expected value) was employed in the research.

The value for the earnings releases was calculated on the basis of individual weighting of a particular stock within

the S&P 500 index, ensuring that all companies received relative merit proportional to their significance for the

equities markets.

The final index presented in this paper is a combination of weighted individual component indexes. This index is a

measurement of the bias prevailing on the market at any point in time. Comparison of the reaction of the market

in different periods of time allows seeing the dynamics of its optimism and pessimism, which is essentially

measuring the immeasurable.


11/23



4. ANALYSIS

4.1 DEFINING RELEVANT ECONOMIC INDICATORS

The first step of our analysis was to separate relevant economic indicators from those that have no impact on themarket. After running correlation analysis between S&P 500 index and all components listed in Table 1, we

determined that the following indicators (Table 2) exhibited a good degree of correlation with the index on the

basis of reliable historical data. The other economic indicators were found to be uncorrelated, derivative or

redundant. Thus, the economic indicators no listed in Table 2 were excluded from the further analysis.

Table 2: Economic Indicators Exhibiting Correlation with S&P 500 Index

Component

Building Permits

Chicago PMI


Existing Home Sales

GDP

Housing Starts


Initial Claims

ISM Index

New Home Sales

Personal Income

Retail Sales

Unemployment Rate

University of Michigan Consumer SentimentIndex

It must be noted that news releases for GDP usually follow in the following sequence: preliminary, initial and final.

Similar situation applies to the University of Michigan Consumer Sentiment Index, which is comprised of

preliminary and revised releases. It was assumed that the market should react just as rationally to revisions as it

should to the actual net-new changes. Therefore, each of the preliminary, initial and final releases was treated as

discrete news pieces expected to impact the market.

The analysis of individual components listed in Table 2 was carried out concurrently, and the results were merged

to form the final index. The calculations of the other components, including earnings, followed exactly the same

path. The complete analysis results will be presented in the section 4.8.

4.2 FILTERS ON THE S&P 500 INDEX

In order to smooth out random index fluctuations in the days preceding and following news releases, we used

closing averages as follows:


12/23



Before_Closethe index value before news release, which is the average of the S&P 500 index closefor the two trading days prior to the news release;

After_Closethe index value after news release, which is the average of the S&P 500 index close oftwo days, including the actual date of the release and the following trading day. The rational for using

two days is the influence of a particular release is statistically greatest on the day of the release, while

effects of the release can be felt on the second day as well, once the dumb money react s to therelease.

The changes in S&P 500 Index were calculated as:

Index =

4.3 NEWSCAST CONTENT

For the purposes of this analysis, we used only the actual values for each news release without offering an

accompanying analytical explanation for the results. As mentioned before, the analysis included value from theprevious news release (PrevValue), the consensus estimates for the current release (ConsValue) and the actual

current release (CurrentValue).

4.4 STEP 1: CALCULATING COMPONENT CHANGES

In the first step, the component changes are calculated. For components expressed as index or percentage, a

direct subtraction was used:

Value = CurrentValue - PrevValue

For components expressed in volume terms (e.g. existing home sales), a percentage changes was used:

Value =

Similarly, the calculations of the expectations vs. actuals were done using the unit of measurement for each

component:

Cons_vs_Value = CurrentValue - ConsValue

Cons_vs_Value =

To ensure consistency in the movement direction, both Value and Cons_vs_Value were selectively adjusted

by -1 to indicate the direction of the news components. For example, rising unemployment is perceived as a

negative development, and thus Value andCons_vs_Value were multiplied by -1, whether rising home sales

is deemed positive, thus Value andCons_vs_Value are taken as is. The total list of adjustments is provided in

Table 3.

Table 3: Economic Indicator Component and Adjustments for -1


13/23



Component Adjusted?

Building Permits No

Chicago PMI No

Durable Goods Orders No

Existing Home Sales No

GDP No

Housing Starts No

Industrial Production No

Initial Claims Yes

ISM Index No

New Home Sales No

Personal Income No

Retail Sales No

Unemployment Rate Yes

University of Michigan Consumer Sentiment Index No

4.5 STEP 2: CALCULATING INDEX SENSITIVITY

The formula used for calculating index reaction to changes in the component was:

Reaction_to_Value = (-1) *

The -1 multiplier was used for highlighting bias as a positive number. Any index and underlying component

movement in opposite directions will create a positive value signalling either positive or negative bias, whethermovements in the same direction will create a negative number indicating no bias. The larger the positive number,

the stronger the bias. For example, if unemployment rises (note: in the previous step in Value obtain a negative

value) and the market falls, then multiplying their ratio by (-1), we obtain a negative value (no bias). However, if

unemployment goes up and so does the market, the value Reaction_to_Value be positive, indicating the presence

of a bias. At this point the nature of the bias (positive or negative) is not yet established.

The second formula, which runs in a parallel to the first one, is the calculation of the market reaction when

comparing news release relative to expectations:

Reaction_to_Cons = (-1) *

After completing Step 2, the market bias for each component becomes visible, identified by the positive

Reaction_to_Value and Reaction_to_Cons values.

4.6 STEP 3: MEASURING BIAS STRENGTH

Reaction_to_Value and Reaction_to_Cons show the presence and degree of bias of the market in its reaction to

news releases.


14/23



To calculate bias leaning (ValueBiasStrength) we applied the following formula:

ValueBiasStrength =

If Reaction_to_Value


15/23



Having identified market leaning on the optimism-pessimism scale and consistency of the bias, it is feasible to

construct a consolidated index for each indicator. Logic suggests that the wider the standard deviation, the weaker

the trend, as values are scattered over a greater range. Therefore, adjusting bias for standard deviation will aid

identifying the magnitude of the bias. The adjustment is done using the following formulas:

ValueBias =

ConsBias =

Finally, we assumed that changes in each economic indicator value carry the same importance as market

expectations for the same indicator. For example, the fact that GDP numbers went up is just as significant as the

fact that the GPD expectations were missed. In this example, the net effect will be zero, as positive news release

will be canceled by the failure to meet or exceed market expectations. The preliminary component index value

was defined as:

Pre_Index = ValueBias + ConsBias

In some cases we observed large spikes caused by unexpected events. To reduce volatility, we applied logarithmic

scale to soften extreme values. The final index for each component was created using the formula:

Index =

* (Ln(ABS(Pre_Index)+Exp(1))-1)

The following are the result for individual components that will comprise the final index (the dotted line

represents index values smoothed by a moving average):

-6

-4

-2

0

2

4

6

LucidityI

ndex

Building Permits

-15

-10

-5

0

5

10

15

LucidityI

ndex

Chicago PMI

-15

-10

-5

0

510

15

20

Lucidity

Index

Consumer Confidence

-4

-3

-2

-1

0

12

3

4

Lucidity

Inde

x



16/23



-5

-4

-3

-2

-1

0

1

2

3

4

LucidityI

ndex

Existing Home Sales

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

LucidityI

ndex

GDP

-6

-4

-2

0

2

4

6

Lucidity

Index

Housing Starts

-1.5

-1.0

-0.5

0.0

0.5

1.0

Lucidity

Index


-8

-6

-4

-2

0

2

4

6

Lucidity

Index

Initial Claims

-15

-10

-5

0

5

10

15

Lucidity

Index

ISM Index


17/23



-5

-4

-3

-2

-1

0

1

2

3

4

5

LucidityI

ndex

New Home Sales

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

LucidityI

ndex

Personal Income

-2

-1

-1

0

1

1

2

Lucidity

Index

Retail Sales

-0.2

-0.1

-0.1

0.0

0.1

0.1

0.2

0.2

Lucidity

Index

Unemployment Rate

-15

-10

-5

0

5

10

15

20

Lucidity

Index

University of Michigan Consumer

Sentiment

-15

-10

-5

0

5

10

Lucidity

Index

Earnings


18/23



4.8 STEP 5: FINALIZING THE INDEX

Having defined individual indexes for each component, a single index combining information from all key economic

indicators can be contracted. The limited information and volatility introduced by the earnings report dictates

creating two indexes for the purposing of selecting the most accurate sentiment indicator.

To combine separate components together we leveraged the A-F scale of economic merit assigned to each

indicator. Weights were assigned based on the indicator importance: A received weighting of 4, B of 3, C of 2 and

D of 1. Shades of the same merit, i.e. + and - signs such as A- and B+, were awarded +/-.30 points added to

the anchor weight. Through a calibration process Earnings received the importance of A. It must be noted that

despite the subjective nature of the weighting assignment done through a calibration process, different weighting

schemes did not result in significant deviations in the final function shape. It strongly suggests the validity and

consistency of evaluating the markets rationality using the proposed approach.

Individual component charts presented above illustrate a slight compatibility issue. Each component showed

different oscillation ranges, and using component as is would cause the one with larger swing amplitude to dwarf

the importance of other components and their contribution to the final index. To level the field we measured each

component in terms of its standard deviation, rather than actual values calculated through the steps outlined

earlier in this paper. It must be noted that for the purposes of this research standard deviations were derived from

the available data spanning over 12 years. Going forward it is imperative to establish the fixed multipliers for each

component to preserve the historical Lucidity Index value, as new data appended to the history of each indicator

will alter the standard deviation calculations. Table 4 lists the weights used for calculating the final index.

Table 4: Economic Indicators Weight in the Final Index

Component Importance Weight

Building Permits B- 2.7

Chicago PMI B 3

Consumer Confidence B- 2.7Durable Goods Orders B 3

Earnings A 4

Existing Home Sales C 2

GDP B 3

Industrial Production B- 2.7

Initial Claims C+ 2.3

ISM Index A- 3.7

New Home Sales C+ 2.3

Personal Income C+ 2.3

Retail Sales A- 3.7Unemployment Rate A 4

University of Michigan Consumer Sentiment Index B- 2.7

The final index is a combined total of weighted individual components grouped by months. Negative and positive

biases are expected to cancel each other out, giving way to the dominant trend. Over time, the trend is expected

to highlight periods of extreme optimism and pessimism.


19/23



Exhibit 1 and Exhibit 2 show the variations of Market Lucidity Index. Exhibit 1 includes the Earnings component,

while the chart on Exhibit 2 is calculated without it. The limited availability of accurate earnings data - the earnings

data for S&P 500 companies collected at the time of the research dates back to 2003 rather than 1998 as other

data series donecessitated confirmation of the final output with a chart that has earnings component removed.

Both charts include a band of acceptable values established at one standard deviation from the mean. Periods with

the index moving above the band mark the times of extreme optimism. During these episodes market downplayedor ignored incoming negative news, reacting positively to the situation that otherwise are deemed negative.

Similarly, deviations below the bottom band indicate periods of strong pessimism when the market went down on

positive news.

Exhibit 1: Market Lucidity Index (with the Earnings component)

Exhibit 2: Market Lucidity Index (without the Earnings component)

-80

-60

-40

-20

0

20

40

60

80

100

120

Index

Market Lucidity Index (Earnings Included)

-60

-40

-20

0

20

40

60

Index

Market Lucidity Index (Earnings Excluded)


20/23



5. PARTING WORDS

Financial markets, while in theory a function of rational behaviour based on unbiased calculations, in reality are

inefficient and frequently irrational. Herding behaviour and tunnel vision of market players remain the key and

inseparable components of the natural and harmonic market functioning. Quite frequently, these behavioural

distortions overshadow sensible valuations, leading to the asset bubbles and subsequent busts. The Market LucidityIndex proposed in this paper is designed for the purpose of measuring the markets reaction to news , which is

indicative of the market bias. Strong bias is expected to signal potential herding behaviours and irrationally fixated

views of the investing crowd.

By itself the Market Lucidity Index carries no forecasting or explanatory value. It is merely a reflection of the

dominant market sentiment with respect to the interpretation of significant economic events. Unlike the P/E ratio,

for instance, which can reflects relative affordability or overvaluation of the markets, or purely technical indicators

like Bollinger Bands and RSI, the index proposed in this paper acts as a litmus test of the behavioural discrepancy

between the markets expected reaction and its actual. In conjunction with other indicators, however, it can be

used a tool to help identify possible issues with the market from the behavioural perspective.

In the ideal world governed by the rational behaviour the index function should oscillate around the 0-mark. Small

index fluctuations will be expected due to the white noise created by the constant stream of global

developments reflected in news releases. However, extreme index deviations from the mean would signal

potential issues that investors must pay close attention to. Lets review some examples of the market behaviour

embedded in the index. Exhibit 3 shows an overlay of the S&P 500 index performance and the Market Lucidity

Index. In late 1999 and early 2000 the market was dominated by the pessimism. Despite seemingly solid

macroeconomic signals, the market reacted negatively to the news releases. It could potentially indicate two

notions: the performance of the economic indicators consistently came below expectations or the market sensed

the trouble ahead and did not fully buy into the positive news.

Exhibit 3: Market Lucidity Index (without the Earnings component) and S&P 500 Performance

600

700

800

900

1000

1100

1200

1300

1400

1500

1600

-60

-40

-20

0

20

40

60

Index S&P


21/23



From the mid-2000 to early 2008, the index fluctuation range narrowed clustering around the 0-level. It strongly

suggests that during this period the market interpreted the news more or less in line with expectations, rising with

positive economic data and falling in line with negative. In March and April of 2008, the market was engulfed by the

very strong optimism, pushing the index up, only see a breath-taking plunge spanning over the next several months.

Commenting on the 2009-2010 rally, the Market Lucidity Index signals a severe disconnect of the market

performance from the underlying fundamentals. The Market Lucidity Index factors both actual changes to theeconomic indicators as well as market expectations set for the news releases. The strong positive bias during that

time suggests that market was possessed by unprecedented optimism (unprecedented based on the available data)

and rallied up not just in situations when economic data showed worsening conditions, but also when economic

readings came below market expectations. The expectancy theory is repeatedly used to explain abnormal market

shifts. Too frequently, when reason and logic fail to explain a particular market reaction, the phase market already

priced it in is used. This, however, is not applicable to the market action exhibited during the 2009-2010 period:

according to the index, the economic readings came consistently below whats was already priced in, as expressed

by the market consensus factored in the Market Lucidity Index.

There are many theories explaining the overly optimistic and irrational market performance in 2009-2010. Often,

the lack of true market volume substituted with the determined one-way HFT algos and FEDs fiddling on the

futures market are singled out as possible drivers behind the rally. Whatever the distortions might be they or the

true market participants pushed the stock indexes higher despite the declines in fundamentals and missed

expectations. This introduces a potentially serious issue for retail investors.

The world of irrational shifts from massive sell-off to extreme rallies is a dangerous place for retail investors to be

in. This paper does not evaluate the market fundamentals not it provides any views on the immediate forecasts for

various markets. We, however, wanted to caution 401K and RRSP holders to stay cautious and conservative. Just

like an emotionally-imbalanced person experiencing sharp mood swings from deepest depression to maniacal

laughs cannot be trusted with helping post-traumatic stress disorder victims; this unpredictable equity market

cannot be trusted with your retirement savings. As the hollow and vigorous optimism replaced the tremendous

market fear, it can easily slide back to the fear again if the public suddenly discovers that it has no ground to stand

on. Until the equity market regains its emotional balance backed by fundamentals, it should be avoided. After all, asJoe Saluzzi of Themis Trading puts it, optimism (which was the driving force behind the rally) is not a strategy.

Certainly, an argument can be made in defence of the market as the leading economic indicator. However, the

leading nature of the market spawns from its ability to read fine signals of improvement or deterioration of the

economic situation. What is commonly felt on the intuitive level by experienced market professionals, the Market

Lucidity Index identifies in quantitative terms. This is the reason for the 2003-2007 market rally not being marked

by any significant changes in the Market Lucidity Index. During that period the market merely reacted to the fine

signals generated by the underlying fundamentals, and index fluctuated around the 0-mark. On the contrary, the

2009-2010 period is associated with widespread optimism. The index suggests that the fine signals, or green

shoots of a typical recovery, did not exist to fuel the rally the way it progressed. It is true that at the initial stages

the recovery was driven by the extremely oversold conditions that developed by March of 2009. However, after

the initial bounce, the rally appears to be supported by optimism only.

If history carries any predictive value, the similar sharp declines and tremendous rebounds of the market sentiment

(business confidence) were observed in 1954-1955, 1958-1959, 1971-1972 and 1982-1983. In all cases it led to the

corrections with the strong optimism of 1971-1972 giving way to the pessimism of the 1973-1975 that brought the

stock indexes down below their pre-rally levels. It is our view that investors should maintain minimal exposure to

equities until the market reaches balance in emotional and fundamental terms.


22/23



If you are interested in receiving further information regarding this research or have any questions for its authors,

please direct your inquiries [email protected]. Thank you for reading.
mailto:[email protected]:[email protected]:[email protected]:[email protected]


23/23


M i th I bl M k t L idit I d P 22 f 22

ACKNOWLEDGEMENTS

We are grateful to Jeff Stuart for his editing assistance.

Jeff Stuart is the President of King James Capital, a private corporation that provides comprehensive services to

publicly listed companies in a professional and cost effective manner. Services include corporate finance, investorrelations and corporate communications. In addition, Jeff provides a range of editing, sales and marketing services

for select newsletter writers in the mining sector. A frequent speaker at mining investment conferences, Jeffs

years of experience as an insider and background as a licensed investment advisor provide him with unique and

balanced insights into the successful development of junior resource companies. More information on Jeffs

business development process can be found atkingjamescapital.com
http://www.kingjamescapital.com/http://www.kingjamescapital.com/http://www.kingjamescapital.com/http://www.kingjamescapital.com/

Measuring the Immeasurable: Market Lucidity Index

Documents

Transcript of Measuring the Immeasurable: Market Lucidity Index