A Maximum Entropy Approach to the Loss Data Aggregation Problem

MAXIMUM ENTROPY APPROACH TO THE LOSS DATAAGGREGATION PROBLEM

Erika [email protected]

joint work withSilvia Mayoral Henryk Gzyl

Department of Business AdministrationUniversidad Carlos III de Madrid

June, 2016

1 / 38

Outline

1 Introduction

MotivationMethodology: Loss distribution Approach

Univariate caseMultivariate case

2 Maximum Entropy Approach

Examples and ApplicationsTheory

3 Numerical Results

Joint marginal density distributionsJoint vector of losses

4 Conclusions

2 / 38

IntroductionMotivation

Banks developed a conceptual framework to characterize and quantify risk, toput money aside to cover large-scale losses and to ensure the stability of thefinancial system.

In this sense, a similar problem appears also in Insurance, to set premiums andoptimal reinsurance levels.

The differences between these two sectors lies in the availability of the data. InOperational risk the size of the historical data is small. So, the results may varywidely.

More precisely, we are interested in the calculation of regulatory/economiccapital using advanced models (LDA: loss distribution approach) allowed byBasel II.

The problem is the calculation of the amount of money you may need in orderto be hedged at a high level of confidence (VaR: 99,9%).

The regulation states that the allocated capital charge should correspond to a1-in-1000 (quantile .999) years worst possible loss event.

It is necessary to calculate the distribution of the losses, and the methodologyused has to take in consideration challenges related with size of the data sets,bimodality, heavy tails, dependence, between others.

We propose to model the total losses by maximizing an entropy measure.

3 / 38

IntroductionLoss distribution Approach (LDA): Univariate Case

Operational risk has to do withlosses due failures in processes,technology, people, etc.

Two variables play a role in operational risk:

Severity (X ): Lognormal, gamma,weibull distributions, Subexponentialdistributions...

Frequency (N): Poisson, negativebinomial , binomial distributions.

S = X1 + X2 + ...+ XN =N∑

n=1

Xn

where S represents the aggregate claimamount in a fixed time period (typically oneyear) per risk event.

Approach used: Fit parametric distributionsto N and X and obtain fS through recursivemodels or convolutionsNo single distribution fits well over the en-tire data set.

4 / 38

IntroductionLoss distribution Approach (LDA): Multivariate Case

Then, each of these loss distributions are further summed among all types of risk, toarrive at the total aggregate loss.

(S1, ...,Sm) = (

N1∑i=1

X1i, ...,

Nm∑i=1

Xmi)

ST =m∑

i=1

Si = S1 + S2 + ...+ Sm

where b = 1, . . . , 8(business lines), l = 1, . . . , 7(event types),m = 8× 7, types of risks in Operational Risk

Dependence structure between risks Si , i.e. choice of a copula model.

5 / 38

IntroductionLoss distribution Approach (LDA): Illustrative Example

Loss distribution Approach (LDA): Illustrative Example

Estimate parametric distributions for the Frequency N and Severity X for eachindividual risk (Maximum-Likelihood Estimation - MLE).

Compound the distributions (Panjer, Convolutions, Fourier, ...). Then, we havefSi

(Univariate Case)

Then, the density fSTof the sum

ST = S1 + ...+ SB

(Multivariate case) can be obtained by a sequential convolution procedure:

1 Derive the distribution of the sum of a pair of values S1 + S2 from thejoint density fS1,S2

(s1, s2) = fS1(s1)fS2

(s2)c(s1, s2), where C is the copulamodel .

2 Apply the convolution integralfS1+S2

=∫

s1fS1,S2

(s1, l12 − s1) =∫

s2fS1,S2

(s2, l12 − s2)

steps (1) and (2) are repeated for the rest of the sum.

6 / 38

IntroductionLoss distribution Approach (LDA): Copulas

It is a method used to introduce dependence among randomvariables.

C (u1, u2, ..., un) = P[U1 ≤ u1,U2 ≤ u2, ...,Un ≤ un]

7 / 38

IntroductionIllustrative Example

200 samples of size 10Aggregation: 7 independent types of risks

What happen when the data is scarce, asin common in banking?

Problems

Parameter uncertainty.

Bad fit in the tails.

Scarcity of data, impossibility to fittails and body separately.

Underestimation of the regulatorycapital charge.

This methodology gives a bad fiteven when re-sampling is analternative.

8 / 38

Maximum entropy approachIllustrative example - size and tail concern

Parametric Approach

3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.00

12

34

Losses

density

True

AVERAGE

Reconstructions

Maxent Approach

Maxent provides a density reconstruction over the entire range of values.

9 / 38

Maximum entropy approachIllustrative example - bimodal concern

(1)(2)

Error (1) (2)

MAE 0.02652 0.01291RMSE 0.03286 0.01647

Table : Errors.

Maxent is able to model asymmetries.

10 / 38

Maximum entropy approachQuestion

How can we use maxent methodologies to model dependencies betweendifferent types of risks in the framework of Operational risk?

11 / 38

Maximum entropy approach

Find a probability distribution P on some measurespace (Ω,F), which is absolutely continuous respectto some (usually σ-finite) measure on Q and on(Ω,F)

maxP

HQ (P) = −∫

Ωρ(ξ)lnρ(ξ)dQ(ξ)

satisfying

P << Q such that EP [AX ] = Y

∫Ω ρ(ξ)dQ(ξ) = 1

This method consist in to find the probability measure whichbest represent the current state of knowledge which is the one

with the largest information theoretical entropy.

12 / 38

Maximum entropy approachJaynes, 1957

This concept was used by Jaynes (1957) for first time as a methodof statistical inference in the case of a under-determined problem.

For example:We rolled 1000 times, a six-sided die comes up with an average of4.7 dots. We want to estimate, as best we can, the probabilitydistribution of the faces.

There are infinitely many 6-tuples (p1, ..., p6) with pi ≥ 0,∑i

pi = 1 and∑

iipi = 4.7

13 / 38

Maximum entropy approachEntropy in Finance

In the last two decades, this concept which has its original role instatistical physics, has found important applications in severalfields, especially in finance

It has been an important tool for portfolio selection, as ameasure of the degree of diversification.

In asset pricing, to tackle the problem of extracting assetprobability distributions from limited data.

14 / 38

Maximum Entropy ApproachGeneral overview

The essence of the Maxent method consist in transform a problem of the type

AX = Y X : Ω→ C

into a problem of convex optimization, by maximizing the entropy measure.Where C is a constraint set of possible reconstructions.Then, we have a

Unique and robust solution

15 / 38

Laplace Transform

In probability theory and statistics, the Laplace transform is defined as expectation ofa random variable.

ψ(α) = E [e−αS ] =

∫ ∞0

e−αs dFS (s) S ∈ IR+

If any two continuous functions have the same Laplace transform, then thosefunctions must be identical.

The Laplace transforms of some pdf’s are not easy to invert and there is nota completely general method which works equally well for all possibletransforms.

16 / 38

Laplace Transform

In probability theory and statistics, the Laplace transform is defined as expectation ofa random variable.

ψ(α) = E [e−αS ] =

∫ ∞0

e−αs dFS (s) S ∈ IR+

If any two continuous functions have the same Laplace transform, then thosefunctions must be identical.

All the information about the problem can be compressed in a set of momentsobtained from the Laplace transform, through a change of variables.

ψ(α) = E [e−αS ] = E [Yα] =

∫ 1

0YαdFY (y) with Y = e−St and Y ∈ (0, 1)

The selection of those moments should be in a way that we use only those thatare the more relevant or informative (Lin-1992 and Entropy ConvergenceTheorem).

17 / 38

Laplace Transform

We want to model fS with S > 0.

When N = 0⇒ S = 0 and we rewrite the Laplace transform as

ψ(α) = E [e−αS ] = P(S = 0) · E [e−αS |S = 0] + P(S > 0) · E [e−αS |S > 0]

ψ(α) = E [e−αS ] = P(N = 0) · E [e−αS |N = 0] + P(N > 0) · E [e−αS |N > 0]

where P(S = 0) = P(N = 0) = po , then

ψ(α) = E [e−αS ] = po · 1 + (1− po ) · E [e−αS |N > 0]

µ(α) = E [e−αS |N > 0] =ψ(α)− po

1− po

ψ(α) and po has to be estimated from the data, or any other procedure.

18 / 38

Input of the MethodologyUnivariate Case

Thus, the problem becomes to determine fS from the integral constraint after achange of variables,

E [e−αS |S > 0] =

∫ 1

0yαj fY (y)dy = µ(αj ), j = 0, ...,K .

Analytical form

ψ(αk ) = E(e−αSt ) =∑∞

n=0(ϕX (t))k pn = G(φX (αk )) with αk = α0/k

Numerical form

ψ(αk ) =1

T

T∑i=1

eαk si with αk = α0/k

where

α0 = 1.5 : fractional value, k = 1, ...,K optimal number of moments.φX (αk ): Laplace transform of X , αk ∈ R+

G(·): probability generation function of the frequenciesψ(αk ): Laplace transform of the total lossesT : sample size.

19 / 38



E [e−αS |S > 0] =

∫ 1

0yαj fY (y)dy = µ(αj ), j = 0, ...,K .

Analytical formFit parametrically the frequency and severity distributions and calculate the Laplacetransform through the probability generation function.Poisson-Gamma

ψ(αk ) = exp(−`(1− ba(αk + b)−a))with αk = α0/k

The quality of the results is linked to how well the data fit to the defineddistributions.

It is not possible to find a closed form of ψ(αk ) for some pdf’s. This isparticular true for long tail pdf’s, as for example the lognormal distribution.

20 / 38



E [e−αS |S > 0] =

∫ 1

0yαj fY (y)dy = µ(αj ), j = 0, ...,K .

Analytical form

ψ(αk ) = E(e−αSt ) =∑∞

n=0(ϕX (t))k pn = G(φX (αk )) with αk = α0/k

Numerical form

ψ(αk ) =1

T

T∑i=1

eαk si with αk = α0/k

where

α0 = 1.5 : fractional value, k = 1, ...,K optimal number of moments.φX (αk ): Laplace transform of X , αk ∈ R+

G(·): probability generation function of the frequenciesψ(αk ): Laplace transform of the total lossesT : sample size.

21 / 38

Input of the MethodologyMultivariate Case

Here we explore how the maxent methodologies can handle with dependenciesbetween different types of risks, when

1 The marginal densities for each risk fS1, ..., fSm are independently modeled, and

the input of the maxent methodology should be adapted to modeldependencies between risks, through a copula model. In order to see thequality of the results we compare with a sequential convolution procedure.

2 Choose the incorrect copula can harm the fit of the distribution. In the casewhere the losses are collected as a joint vector (where the dependencies areincluded), the maxent techniques can find directly the density distribution.We compare the results with a sequential convolution procedure with a correctand incorrect copula to see the differences.

22 / 38

Input of the MethodologyMultivariate Case

(1) We can aggregate dependencies to our input, knowing each fSi

ψ(α) = E [e−αk (S1+S2+...+SB )] =

=

N−1∑i=1

e−(s1i +s2i +...+sBi )αk f (s1i , s2i , ..., sBi )∆s1∆s2...∆sB

where N is the number of partitions used in the discretization and

f (s1, s2, ..., sB ) = c[F1(s1), ...,F1(sB )]B∏

i=1

fSi(xi )

the joint distribution, c is the density of the copula model C , and fS1, ..., fSB

aremarginal densities.

(2) Simply ψ(α) = 1T

T∑i=1

e−αk (s1i +s2i +...+sBi )

where T is the sample size.

23 / 38

Maximum Entropy Methods

max H(f ) = −∫ 1

0fY (y)lnfY (y)dy

SME approach. Find the probability density on [0,1]

ψ(α) =

∫ 1

0yαk f (y)dy = µ(αk ) with Y = e−S

where µ = ψ(αk )−P(N=0)1−P(N=0)

MEM approach: Extension of the SME approach, allows to include a referencemeasure Q, which is a parametric distribution.SMEE approach: Extension of the SME approach when we assume that thedata has noise. ∫ 1

0yαk f (y)dy ∈ Ck = [ak , bk ] with Y = e−S

These methods consist in to find the probability measure which best representthe current state of knowledge which is the one with the largest information

theoretical entropy.24 / 38

Standard Maximum Entropy Method (SME)

In general, the maximum entropy density is obtained by maximizing the entropymeasure

Max H(f ) = −∫ 1

0fY (y)lnfY (y)dy

satisfying

E(yαk ) =

∫ 1

0yαk fY (y)dy = µαk , k = 1, 2, ...,K with K = 8

∫ 10 fY (y)dy = 1

whereµk : k-th moment, which it is positive and it is a known valueK = 8: number of momentsFractional value: αk = α0/k, α0 = 1.5

25 / 38

Standard Maximum Entropy Method (SME)When the problem has a solution can be expressed in terms of the Lagrangemultipliers as

f ∗Y (y) =1

Z(λ)exp

(−

K∑k=1

λk yαk

)= exp

(−

K∑k=0

λk yαk

)the normalization constant is determined by

Z(λ) =

∫Ω

exp

(−

K∑k=1

λk yαk

)dy

Then it is necessary to find λ∗, that is the minimizer of the dual entropy that isin function of the Lagrange multipliers λ and is given by

H(λ) = lnZ(λ)+ < λ, µ >=∑

(λ, µ)

Basically it is a problem of minimizing a convex function and there have to re-duce the step size as it progresses (Barzilai and Borwein non-monotone gradientmethod)

f ∗Y (y) =1

Z(λ∗)exp

(−

K∑k=1

λ∗k yαk

)Y ∈ (0, 1)

26 / 38

Standard Maximum Entropy Method (SME)1 The starting point is

∫∞0 e−αs dFS (s) = µk , S ∈ (0,∞)

2 Make a change of variables, setting Y = e−S(N), Y ∈ (0, 1),

3 Find a minimum of the dual entropy that is in function of λ,

minλ

∑(λ, µ) = lnZ(λ)+ < λ, µ >

where

Z(λ) =

∫ 1

0e−

∑Kk=1 λk yαk

dy .

4 The solution

f ∗Y (y) =1

Z(λ∗)e−

∑Kk=1 λ

∗k yαk

= e−∑K

k=0 λ∗k yαk

Y ∈ (0, 1)

5 Return the change of variables

f ∗S (s) = e−s f ∗Y (e−s ), S ∈ (0,∞)

27 / 38

Numerical ResultsMultivariate Case





28 / 38

Numerical Results

To test the methodology we consider different combinations of frequencies andseverity losses.

We use a sample large enough for not to worry about the effect of the size inthe results.

We use several methods to verify the quality of the results: L1 & L2 distances,MAE & RMSE distances, visual comparisons, and goodness of fit tests.

MAE =1

T

T∑n=1

|F (xn)− Fe (xn)|

RMSE =

√√√√ 1

T

T∑n=1

(F (xn)− Fe (xn)

)2

RMSE is more sensitive to outliers, because this measure gives a relatively highweight to large errors. So, the greater the difference between MAE and RMSE,the greater the variance of the individual errors in the sample.

29 / 38

Numerical Results

S

De

nsity

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8

01

23

4

(a) S1

SD

en

sity

1.5 2.0 2.5 3.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

(b) S2

S

De

nsity

4.0 4.5 5.0 5.5 6.0 6.5 7.0

0.0

0.5

1.0

1.5

(c) S3

Figure : Losses for each line of activity, reconstructed by SME

ErrorS MAE RMSES1 0.0054 0.0072S2 0.0241 0.0282S3 0.0061 0.0071

30 / 38

31 / 38

ErrorCopula MAE RMSE

Independent 0.027 0.039Gaussian, ρ = 0.5 0.004 0.005Gaussian, ρ = 0.8 0.004 0.005

t − student, ρ = 0.7 , ν = 10 0.004 0.00532 / 38

Numerical ResultsMultivariate Case





33 / 38

Gaussian copula with weak Negative correlation

S

Density

6.5 7.0 7.5 8.0 8.5 9.0

0.0

0.5

1.0

1.5

SME (Gaussian)

Convolution (t−student)

Convolution (Gaussian)

Error SME Convolution(Gaussian) Convolution (t-Student)

MAE 0.007989 0.01071 0.01430RMSE 0.009605 0.01264 0.01652

Table : Errors SME approach.

34 / 38

Computation of the regulatory capital

VaR

Approaches Errors Conf. Interval 95%γ SME Conv. Empirical SME err. Conv.err. VaRinf VaRsup

0.900 7.237 7.293 7.236 0.001 0.057 7.212 7.2630.950 7.399 7.293 7.365 0.034 0.072 7.309 7.3890.990 7.682 7.569 7.658 0.024 0.089 7.516 7.6890.995 7.803 7.707 7.719 0.084 0.012 7.595 7.8560.999 8.175 8.534 8.601 0.426 0.067 7.689 8.926

Table : Comparison of VaR for the SME & Convolution approach

35 / 38

Computation of the regulatory capital

TVaRγ(S) = E [S |S > VaRγ ]

TVaR

Approaches Errors Conf. Interval 95%γ SME Conv. Emp. SME err. Conv.err. TVaRinf TVaRsup

0.900 7.439 7.404 7.419 0.020 0.015 7.336 7.5360.950 7.571 7.514 7.549 0.022 0.035 7.415 7.7350.990 7.892 7.837 7.920 0.028 0.083 7.551 8.4430.995 8.052 8.089 8.047 0.005 0.042 7.578 8.9260.999 8.529 8.758 8.334 0.195 0.424 7.658 8.926

Table : Comparison of TVaR for the SME & Convolution approaches.

36 / 38

Conclusions

In this work we present an application of the maxent methodologies toOperational Risk. We showed that this methodology can provide a good densityreconstruction over the entire range, in the case of scarcity, heavy tails andasymmetries, using only eight moments as input of the methodology.

This methodology gives the possibility to obtain the density distributions fromdifferent levels of aggregation and allow us to include dependencies betweendifferent types of risks.

We can joint marginal densities obtained from any methodology and givethem any relation orwe can obtain the joint distribution directly from the data and avoid badestimations

The estimation of the underlying loss process provides a starting point to designpolicies, set premiums and reserves, calculate optimal reinsurance levels andcalculate risk pressures for solvency purposes in insurance and risk management.Also, this is useful in structural engineering to describe the accumulated damageof a structure, just to mention one more possible application.

37 / 38

QUESTIONS, COMMENTS

38 / 38

A Maximum Entropy Approach to the Loss Data Aggregation Problem

Economy & Finance

Transcript of A Maximum Entropy Approach to the Loss Data Aggregation Problem