NLP&CC 2013 1

Automatic Assessment of Information Disclosure

Quality in Chinese Annual Reports

QIU Xinying, JIANG Shengyi, DENG KebinCISCO School of Informatics

Guangdong University of Foreign Studies

NLP&CC 2013 2

Outline• Background • Methodology and Design• Results and Analysis• Conclusions

NLP&CC 2013 3

Research Background• Corporate information disclosure:

– Annual reports; Quarterly reports – Earnings forecast; press release– Financial news

• Why study them?– Forecast of companies’ performance– Investment decisions– Regulations and management

NLP&CC 2013 4

Research Background• All about ENGLISH documents; • No research is conducted about

Chinese information disclosure

NLP&CC 2013 5

Research Background• Research perspectives:

– Document level• Build predictive models with disclosure

documents for stock return forecasts– Tsai et al. (ECIR ‘ 13); Lin et al. (ACM TOMIS ‘ 11);

Balakrishnan et al. (EJOR ‘ 10); Kogan et al. (NAACL ‘ 09)

– Feature level• Risk; Tone; Readability; Forward looking

statement– Feldman et al. (RAS ‘ 10); Lehavy et al. (TAR ‘ 11);

Li (JAE ‘ 08); Li (JAR ‘ 10);

NLP&CC 2013 6

Our work• General goal:

– to pave the way for the study of Chinese information disclosure from text mining perspective

NLP&CC 2013 7

Our work • In this work:

– To build automatic system to evaluate Chinese disclosure quality

– To explore and mine features factors for better understanding and utilization of Chinese reports

• More specifically:– Multi-class classification system– Readability analysis with regression

NLP&CC 2013 8

Methodology• Four-class classification for

automatic quality evaluation

NLP&CC 2013 9

Methodology• Chinese Readability index

NLP&CC 2013 10

Methodology• Regression analysis about

readability and analysts following

NLP&CC 2013 11

Results and Analysis• 4-class quality classification:

• About 10% better than the equivalent classification of English reports with stock return for class standards

NLP&CC 2013 12

Results and Analysis

NLP&CC 2013 13

Results and Analysis

NLP&CC 2013 14

Results and Analysis• Analysts effort in following annual

reports is negatively associated with the level of difficulty in reading the reports. In other words, easier to read annual reports attract more attention from analysts in their evaluation.

• Results different from counterpart analysis with English reports

NLP&CC 2013 15

Conclusions• Our model for overall four-class

classification achieves better performance to the extent of classification accuracy than the counterpart research on English reports.

• Distinguishing between excellent versus fail quality reports is much more efficient than between good and pass quality reports.

NLP&CC 2013 16

Current Work

NLP&CC 2013 17

Current Work

NLP&CC 2013 18

Current Work

NLP&CC 2013 19

Thank you!