How Much Credit Crdt Rsk Mdls

download How Much Credit Crdt Rsk Mdls

of 15

Transcript of How Much Credit Crdt Rsk Mdls

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    1/15

    Banks

    8 May 2007

    www.fitchratings.com

    Analys ts

    Specia l Pro jects Group

    Gary van Vuuren+44 207 417 [email protected]

    Krishnan Ramadurai+44 20 7417 [email protected]

    Quant i ta t ive F inanc ia l ResearchGreg Gupton+1 212 908 [email protected]

    USA

    Eileen Fahey+1 312 368 [email protected]

    EMEA

    Ian Linnell+44 20 7417 [email protected]

    Asia

    David Marshall+8 52 2263 [email protected]

    Algo r i t hm icsKim Olson+1 212 612 [email protected]

    Diane Reynolds

    +1 416 217 [email protected]

    Table of Content s

    Executive Summary ......................... .......1Assessment of Credit Portfolio RiskModels in the Ratings Process........... 1

    Introduction.............................................2Credit Risk Models andRisk Management...............................2Prime Drivers of Credit Portfolio RiskModels................................................3Credit Portfolio Risk Models..............5Assessing Quality of Credit RiskModels................................................6Example..............................................6

    Appendix 1 ..................... ....................... 11Credit-Worthiness ModellingApproaches.......................................11Structural Approaches ...................... 11Reduced form Approaches ............... 11

    Appendix 2 - Possible Questions...........12

    References.............................................15

    Execut ive Summary

    This paper addresses the role of credit risk within internal eco-nomic capital models in assessing risk appetite and capital ade-quacy in Fitchs assignment of credit ratings. Other papers ad-dressing related issues such as the role of market, operational risksindividually and in combination with credit risk in the context ofeconomic capital models will follow.

    Regulatory capital requirements are minimum capital requirements.The amount of capital retained in excess of regulatory require-ments will materially influence the credit rating assigned.

    This paper outlines characteristics needed in order to place mate-rial reliance on their outcome.

    Assessment of Credi t Port fo l io R isk Models in t he

    Ratings Process

    Fitch Ratings considers the institutions own economic capital andregulatory capital models as a factor in its ratings process of banks.

    Factors discussed throughout this paper will influence the degreeof comfort Fitch will derive from an institution's credit risk model.We will place a higher weight on models with the following char-acteristics:

    Board and Management Overs ight o f Credi t

    R isk Models Board and senior management should understand and endorse theuse of credit risk models. The Board should ensure that the modelis integrated into active management of the company. The outputsand reports of the models should be used to assign and track risktolerance and associated capital levels. Management should re-ceive regular reports. The Board should require and be apprised ofthe independent validation of models. Individuals involved inmodel development should not also be the end users. If this is notpossible, an external third party should validate the model.

    A High Degree o f Transparency t o the Agenc y

    The more information provided by the institution (eg detaileddocumentation of model construction and implementation), thegreater the weight Fitch would expect to apply to the institution'sown model output. This assumes that models address key creditrisks consistently.

    Management should be sufficiently well-versed in their models tobe able to explain and account for fundamental areas of similaritiesand differences between model outputs and both accounting andregulatory frameworks.

    Criteria Report How muc h Cred i t in Cred i t

    Risk Models?

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    2/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    2

    A Proven Track Record

    Models that have been consistently employed to as-sess risk over a period of time, and where manage-ment have a history of decision making and manag-

    ing their business using the output provided by themodels will be assigned higher weights. Credit riskmodels that are in place but not integrated with theinstitutions risk management and decision-makingframework are, in general, ineffective.

    Appropr ia te Cont ro l o f t he Model

    As models can be powerful tools within a banksbusiness, model controls are important and are a keyingredient of Fitchs assessment. Models should beembedded within the business. Change control anddocumentation standards should ensure model integ-rity.

    Qual ity o f Model

    Models should be chosen that provide a strong fitbetween their capabilities, assumptions and the prop-erties of the portfolio. Some contemporary credit riskmodels employ a single-factor model frameworkand/or assume an infinitely fine-grained portfolio.They are appealing because of their simplicity. Suchmodels may be appropriate for highly diverse portfo-lios, as found in retail banking or SME divisions, butcan only rarely be applied to corporate portfolios.The tractability of credit risk models is lost whenless finely-grained portfolios or portfolios which

    intrinsically depend on a large number of disparatemarket factors are to be modelled. Thus, it is essen-tial that a suitably sophisticated model be employed.

    Models should be updated and shortcomings ad-dressed as the industry continues its enhancements.Appendix 1 provides a brief overview of commoncredit risk model approaches.

    Val ida t ion

    While validation processes and methodologies arethe prime responsibility of the institutions them-selves, Fitch will evaluate and assess the process and

    techniques deployed and attempt to identify bestpractice amongst institutions.

    Ongoing Model Review

    Models should be subject to regular periodic audits,continuous testing and benchmarking. Relatedchange control procedures are also important to en-sure that only reasonable changes are made withoutsacrificing model integrity.

    In t roduct ion

    Fitch recognises the advances made by banks in riskmanagement and in particular the increasing use of

    credit portfolio risk models for the measurement andmanagement of credit risk as part of the overall eco-

    nomic capital management framework of these or-ganisations. The growing popularity of these modelsis only expected to accelerate with the establishmentof a new regulatory framework for assessing bank

    risk-based capital, better known as Basel II. Basel IIadapts many of the concepts and techniques used bylarge financial institutions in their own internal creditrisk models for supervisory purposes.

    The agency believes that the anticipated convergencebetween economic capital and regulatory capitalframeworks has been a positive impetus to furtherrefine and more broadly spread advanced risk meas-urement and management practices such as creditportfolio risk modelling.

    The assessment of credit risk models and their outputrepresents an important part of Fitchs overall viewof an institutions risk appetite and profile within theagencys rating analysis. A number of institutionsuse standard vendor models available in the marketto measure and manage credit risk. There is an in-creasing trend by many institutions to (1) attempt toseparate the conceptual model from its implementa-tion, (2) internally develop credit risk models built tomeet their organisations unique requirements, and(3) access greater diversification benefit through the

    joint analysis of multiple asset classes.

    This report identifies the principal drivers of creditrisk models in the measurement of credit risk and theinfluence of these drivers on the economic capitalcharge, as well as the impact of these drivers if ap-plicable in determining the capital charges under thenew regulatory capital framework (Basel II internalratings-based approach). The report uses a sampleportfolio of loans as an illustration to compare theinfluence of these drivers in determining capitalcharges under economic and regulatory capitalframeworks. In addition, the report also outlines theagencys opinion on the use of these credit risk mod-els in the measurement of credit risk and the associ-ated capital charge and how it will factor this into itsrating process.

    Credit Risk Models and Risk Management

    Fitch believes that credit risk models form a veryimportant part of risk management and are a vitalinput into the pricing of products and services. Theunderstanding of risk on an aggregate basis and anassessment of how these risks associated with singleborrowers interact with each other is very difficult toachieve in the absence of some kind of risk model,especially where risks are complex. It is also theagencys opinion that while measuring risks is a ne-cessity, to achieve full economic benefit from themodels, their output should be an influencing factorin decision making within an institution.

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    3/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    3

    A credit risk model can be instrumental to an organi-sations management: it aggregates individual risksthat reside in separate silos and enables an assess-ment of the overall credit risk faced by an organisa-

    tion. Credit risk models provide important data andinformation to aid strategic decision-making. It iscritical to understand the prime drivers and how theycombine to influence the final output of these models.

    Prime Dr ivers of Credit Port fol io Risk

    Models

    Probabi l i ty of Defaul t : PD

    Most large financial institutions use risk ratings todistinguish among obligors of different credit quality.However, different rating systems accomplish thistask in different ways. Rating systems can employ as

    few as four or five to as many as 30 categories ofrisk, to classify corporate and individual customers.In general, it is noted that many institutions use rat-ing systems which are characterised by one or moreof the following three elements: (i) a traditionalmethod which combines financial and other charac-teristics of the customer into a relatively subjectiveapproach to determining ratings; (ii) vendor-suppliedcredit scoring models; or the use of (iii) internallydeveloped credit scoring models. While element (i)is commonly found in rating systems for corporatecustomers, elements (ii) and (iii) are predominantlyfound in the rating systems for retail and small and

    medium-sized enterprise (SME) customers.

    In addition, it is necessary to understand the dynamicproperties of a rating system (i.e. whether they aredesigned to capture "through-the-cycle" (TTC) or"point-in-time" (PIT) conditions) and the relation-ship with default probabilities (PD). Under PIT rat-ing systems, obligors are assigned risk buckets withunique PDs based on best available informationabout their credit quality and, as this changes, obli-gors are rapidly transitioned to new risk buckets withdiffering PDs. By contrast, under TTC systems, ob-ligors are assigned to rating grades and PDs based on

    evaluations of their abilities to remain solventthrough a business cycle which includes a severestress event. Because of the weight placed on stressconditions rather than current conditions, PDs ofTTC ratings tend to change less often than PDs ofPIT ratings and therefore tend to be more stable overthe business cycle. In practice as it is very difficult tocharacterise PDs as being pure TTC or PIT and ascapital calculations are sensitive to PDs (PIT orTTC) it is critical to understand the process by whichPDs are determined.

    As PD is the likelihood of a borrower defaulting

    within a defined time frame, all institutions,irrespective of the rating systems used, will need to

    validate their PD estimates using historical defaultinformation. The definition of default used by aninstitution is a critical starting point since the pa-rameter is based on a default event.

    Def in i t ion of Defaul t

    Many unconditional credit risk models assume thatobligor default is triggered by a change of the obli-gors asset value, ie obligors default if their institu-tions value falls below a pre-specified thresholdpoint usually taken to be the point at which an insti-tutions asset value equals its liabilities. Obligorsasset values are commonly modelled by a common,standard, normally-distributed factor component andan idiosyncratic standard normal noise or residualcomponent. It should be noted that this definition ofdefault is fundamentally different from the definition

    used by the accounting framework and an analysis ofdifferences between the two is essential for a properunderstanding of how credit risk models work.

    From a regulatory perspective, default events arise ingeneral from the non-payment of principal or interestfor 90/180 days or when it seems unlikely that obli-gations will be met. However, an institution mayalso characterise a loan to be in default where bor-rowers are either in violation of covenants or areexperiencing severe cash-flow problems. As thedefinition of default can vary across institutions andasset classes, it is critical to understand the rationale

    behind the default definition adopted by an institu-tion and make an assessment of its application acrossdifferent asset classes. While estimating PDs, it isalso critical to understand the assessment horizon ortime-frame taken into consideration.

    For example, the use of a one-year PD implies as-sessment of default over the next 12 months and isstrictly related to the rating of the counterparty or aspecific deal. This can be contrasted with cumulativeor multi-year PDs, where the assessment of default isover a number of years. One common approach tomodel these PDs is the introduction of a migration ortransition matrix to account for the possibility thatratings change over time.Loss Given Defaul t : LGD

    The LGD is a measure of the expected loss that thebank will experience per unit of exposure should itscounterparty default, and is expressed as a percent-age. Therefore, the definition of default for LGDpurposes should be consistent with the definitionused for calculating PDs. Although each borrowermay have only a single borrower rating reflectedby a single probability of default different expo-sures to the same borrower will result in different

    facility-specific LGD profiles.

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    4/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    4

    The ex ante loss on an exposure if it were to defaultis a random variable, whose value is unknown untildefault occurs. Indeed, since the actual net cash-flows on a defaulted loan are not known until a

    workout is complete, there is uncertainty surround-ing the LGD even at the time of default. The ex-pected LGD may be derived from the market price ofdefaulted debt, from a banks internal workout andrecovery data, or from analysis of consortium-provided workout and recovery data.

    Outside the consumer and small business lendingareas, there is very limited historical data with whichto estimate LGDs. This data problem is further com-pounded by the fact that LGDs normally will dependon the loans seniority and collateral (if any), as wellas on characteristics of the borrower, such as indus-

    trial sector and country.

    Where internal workout data is used, recovery cash-flows net of workout costs are discounted at theirpresent value back to the default point. The rate atwhich these cash-flows are discounted should, inprinciple, be the rate appropriate for the risk of thedefaulted asset. However, this rate is not directlyobservable, nor is it easy to estimate because data isscarce. Available data for estimation may be influ-enced by issues such as liquidity. Difficulties alsoarise in identifying a relevant market for all the vari-ous transactions a bank undertakes.

    For these reasons a common choice for discountingis the contractual interest rate which has the advan-tage of being consistent with accounting rules. Al-though other valid choices exist and are appropriateto use, the relevant interest rate chosen must be justi-fied and the institution must demonstrate that theresultant LGD estimates are reasonable.

    The lack of robust data on which to base LGDsacross a range of borrower and transaction types is acommunal and significant problem in all credit riskanalysis. Only a minority of institutions employ his-torical data spanning more than five years: external

    sources of these data are growing, but remain notentirely independent from the banks own uniqueloss history since they tend to be based on sponsoredconsortium structures.

    Exposure at Defaul t : EAD

    Exposure at Default (EAD) is the extent to which abank is exposed to a credit or counterparty in theevent, and at the time, of that counterparty's default.

    EAD is a difficult parameter to model for some facil-ity types although conceptually easy to understand.First, there is the question of how one defines expo-

    sure: the nominal sum outstanding, the market valueof exposure just prior to default, or the market value

    of the exposure just prior to default but assuming itwere risk-free. Actually, because EAD and LGD arebroadly fungible, this issue is often addressed underLGD rather than under EAD, with EAD defined to

    be outstanding balance sheet exposure.

    Second, EAD, in many cases depends on the actionsof the counterparty (drawdown when default is im-minent) and the bank (withdrawal of limits, tightercovenants). Although there is the possibility thatcurrent outstandings may be reduced prior to default,it is reasonable to hold capital against all drawn ex-posures. In addition, a positive EAD is expected forall un-drawn commitments and advised limits, in-cluding unconditionally and immediately cancellablelines of credit.

    Third, EAD may be dependent on other factors. Forexample, in the case of over-the-counter derivativesand securities financing transactions, market ratesplay a role in determining the market value of thepositions today and at the time of default. WhileBasel II allows for varying levels of sophistication intreatment of such exposures, internal models arebecoming ever more sophisticated, particularly inlarge institutions.

    Matur i ty : M

    The maturity of a derivative, bond or loan is one ofthe key factors affecting credit risk. All else beingequal, the shorter the maturity, the less the corecredit risk. Banks can deny credit, raise prices ordemand greater protection (in the form of collateralor seniority) in the quest to limit future losses to cus-tomers whose financial conditions have deteriorated.

    Corre la t ion :

    In most banks, the default of a single counterparty isunlikely to be a drain on capital. Rather it is a seriesor group of defaults that poses a threat. Correlationis the driver that determines the propensity of de-faults to group together and hence is an importantfactor in portfolio analysis. A key differentiator be-

    tween credit risk models is in the description andcalibration of correlations.

    Closely related to correlation is the concept of con-centration. Generally, credit portfolio risk modelsare strongly influenced by the property of portfolioinvariance. Idiosyncratic risks (associated with indi-vidual exposures, and hence the source of concentra-tion) in large portfolios of small exposures tend tocancel one-another out and only systematic risks thataffect many exposures have a material effect on port-folio losses.

    In many credit risk models, all systematic risks thataffect all borrowers to a certain degree (such as in-

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    5/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    5

    dustry or regional risks) are commonly modelledwith only one systematic factor. This is the case, forexample, in the model underlying the Basel II stan-dards. The single factor needed in the model may be

    interpreted as reflecting the state of the economy.The degree of the obligors sensitivity to this sys-tematic risk factor can be expressed by the correla-tion between the factor and the value of the obligorsassets. So, the correlations could be described as thedependence of a borrowers asset value on the gen-eral state of the economy; all borrowers are linked toeach other by this single risk factor.

    Asset correlations essentially indicate the degree ofco-movement between the asset value of one bor-rower and the asset value of another. Asset correla-tions also influence the shape of the risk weight for-

    mulas. They are asset-class dependent, because dif-ferent borrowers and/or asset classes show differentdegrees of dependency on the overall economy.Time series of default frequencies have been used todetermine default rates as well as correlations be-tween borrowers. Analysis of these time series re-veals two systematic dependencies:

    1. Asset correlations decrease with increasing PDsas indicated by both empirical evidence and in-tuition. Intuitively the higher the PD, the higherthe idiosyncratic (individual) risk components ofa borrower. The default risk depends less on theoverall state of the economy and more on indi-vidual risk drivers.

    2. Asset correlations increase with firm size, againas indicated by both empirical evidence and in-tuition. Although empirical evidence in this areais inconclusive, it again appears intuitive that thelarger the firm, the higher its dependency uponthe overall state of the economy, and vice versa.Smaller firms are more likely to default for idio-syncratic reasons.

    Asset correlations are commonly an intermediatestep to derive the default correlations that are used in

    some credit models. Default correlation is a functionof: the asset correlation and the PDs of the two obli-gors that are being correlated. Asset correlationtranslates into higher default correlations when one(or especially both) obligor PDs are high.

    It is also important to recognise that correlation val-ues used in financial institutions in their economiccapital models may well differ from those used inregulatory capital models. Typical reasons for thedifferences include factors such as stated conserva-tism in the Basel II standards and the use of multi-factor models for economic purposes versus single-

    factor models for regulatory purposes.

    Empirical evidence has so far been contradictoryregarding both the relationship between default cor-relation and PD, and the average level of correlationacross geographic regions. Assessing the magnitude

    of (and reasons for) these correlation differences andpatterns will, therefore, be a key consideration forFitch in the ongoing quest for data transparency andintegrity.

    Credit Port fol io Risk Models

    Credit risk modelling has developed rapidly in therecent past and is now firmly established as a keycomponent of risk management systems in financialinstitutions. A wide variety of credit risk modelshave evolved, differing in methodologies and fun-damental assumptions. For example, the definition ofcredit losses (default models define credit losses as

    loan defaults, while mark-to-market or multi-statemodels define credit losses as ratings migrations ofany magnitude). In the end, all the identified driversare combined in a credit risk model to generate a lossdistribution, which in turn determines the portfoliocapital charge.

    Generically, credit risk models can be classified aseither conditional or unconditional. Unconditionalmodels do not adjust expectations for cyclicalchanges in economic conditions, while conditionalmodels adjust explicitly.

    All models are conditional to some extent, since allmodels seek to incorporate some current informationabout the quality of credit of each borrower and ofeach credit facility. It is, however, still possible todistinguish between unconditional and conditionalmodels.

    In conditional models, explicit adjustments could lagbehind the economy by several months due to thetime required to procure and process econometricdata. However, such models are often favoured bythe market for their explicit handling of economicconditions on the basis that ideal credit modelsshould account for changes in the economic envi-ronment, and not assume constant economic parame-ters.

    The common goal of all credit risk models is to fore-cast the loss distribution function that may arise froma banks credit portfolio. Since credit defaults orrating changes are relatively rare events and sincedebt instruments comprise set payments that limitpotential returns, the loss distribution is generallyasymmetric, ie skewed towards zero, with a longright-hand tail. Although institutions do not employthe entire loss distribution for capital purposes, dif-ferent parts of it can be used in support of different

    decision-making processes (e.g., pricing) so, credit

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    6/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    6

    risk models typically characterise the full distribu-tion.

    There are a number of commercial credit risk modelsin addition to those developed internally. To effec-tively employ the results in risk practice and othermanagement activities, it is essential for banks tofully understand how these credit models function.Further, it is important for the bank to be able toprovide sufficient detail for Fitch to be able to under-stand, assess and contrast the particular credit riskmodel being used.

    Assessing Qual i ty of Credit Risk Models

    Two important components need to be addressed aspart of the assessment process, and certainly beforecredit risk models can be used to determinerisk-based capital requirements.

    1. Model input quality. The need to establish con-sistent and accurate exposure measurement forthe chosen credit rating standard is very impor-tant. Much work has been conducted regardingthe difficulties involved in creating andmaintaining accurate internal ratings systemsand although these present challenging issues,they can be addressed by internal and externalqualitative monitoring procedures.

    2. Model specification and validation. The con-struction of the loss distribution function is pos-

    sibly the most challenging aspect of credit riskmodelling. Changes in credit value are due toseveral factors, such as correlations betweenportfolio assets, movements of credit spreadsand changes in individual loans credit status.As a result, a variety of modelling assumptionsand parameter values are required for the con-struction of a credit risk models forecasted dis-tribution. The lack of sufficient historical per-formance data, however, limits the testing of thevalidity of these model components. A relativelylarge number of observations are required to ac-curately evaluate any distribution forecast. Thisissue is of less concern for evaluating distribu-tion forecasts generated by marketValue at Risk(VaR) models since such models generate dailyforecasts of market portfolio changes which canthen be evaluated relative to the actual portfoliochanges. Whilst some 250 observations (oneyear of daily trading data) are currently used inthe regulatory evaluation ofmarketVaR models,it is difficult to gather a large number of ob-served credit losses with which to evaluatecredit risk models since these have much longerforecast horizons typically one year. As a re-sult, one year generates a single observation ofcredit losses, so 250 years would be required to

    collate an equivalent number of observations asspecified for market risk models.

    3. Comprehensiveness. It is important to theassessment process to clearly identify the risksthat are being considered and those that havebeen ignored by the credit risk model. Whilethere are many potential inclusions and exclu-sions, two of the most commonly excluded areconcentration risk and migration risk. Consideras an example, concentration risk: Name con-centration in loan portfolios arises from either adearth of borrowers or when loan amounts areasymmetrically distributed. The credit portfoliorisk model adopted for the Basel II Internal Rat-ings-Based (IRB) approach does not account forname concentration: it assumes that the banks

    portfolio is perfectly fine-grained, meaning thatidiosyncratic risk has been fully diversifiedaway, so that economic capital depends only onsystematic risk. As a result, each loan consti-tutes only a small fraction of the total portfolioexposure. True bank loan portfolios are seldom if ever perfectly fine-grained. The asymp-totic assumption, while approximately valid forlarge portfolios, may be invalid for portfolios ofsmaller or more specialised institutions. Wherematerial name concentrations of exposure exist,the residual of undiversified idiosyncratic risk inthe portfolio can be significant. Fitch will expect

    financial institutions to have a process in placeto identify, measure, manage and mitigate con-centration risk.

    Example

    Appl ica t ion o f Cred i t R isk Model

    Methodo logy to Loan Por t fo l ios

    A simple Excel-based single-factor model was con-structed to examine how these primary drivers influ-ence model output. The model was also flexed to testsensitivity of model output to changes in values ofprimary drivers, and to compare it to the regulatory

    model.

    It is important to note that construction of afull-blown credit risk model is not the purpose of thispaper. The model employed here (referred to as theBasic Model) therefore, makes many assumptionsabout the structure of credit portfolios. It is thus rela-tively simple, aiming to illustrate the principles onwhich these models operate rather than focussing onthe details.

    The data used are hypothetical corporate loan portfo-lios (which approximate real life bank loan portfo-

    lios) and were characterised by the parameters (usedas input to the single-factor model) specified below.

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    7/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    7

    Further, the Basic Model assumes that a loan's EADis simply its current outstanding exposure. In reality,a loan's EAD depends on several factors, e.g., cou-pon payments, coupon payment frequency, discount

    rates, and so on all ignored in the interests of sim-plicity in this demonstration. Similarly any morecomplex instruments are ignored in this analysis.

    The results are shown in the figures below. Some20,000 simulations were run for each scenario inves-tigated. This provides a confidence in the results to+/-5%.

    Throughout this study, the loans were grouped byPD and LGD classes. For simplicity, both the PDsand LGDs were gross-exposure-weighted within aparticular class. Gross-exposure-weighted hererefers to a scheme which weighs the variable inquestion by exposure only, not by the product ofexposure and LGD (or net exposure). Weighting thePD using the net exposure method would result inmean-invariant portfolio losses, and so is a techniquethat Fitch understands is widely used in credit port-folio analytics.

    Port fo l io Resu l ts

    In addition to a base case, eight scenarios were ex-plored for the portfolio detailed in the table below.Each scenario was designed to investigate the sensi-tivity of the expected and unexpected losses tochanges in the relevant credit risk component(hold-ing other constituents fixed). In each case, all otherelements remain the same as for the base scenarioapart from the stressed component.

    The effect of changing a single parameter in thismanner can be important to fundamental analysis, aswe provide herein. However, we recommend exer-cising caution: there are complex interactions be-tween the portfolio content, the assumptions and theconfidence level at which the results are measured.The complexity of this interaction means that the

    ordering of the significance of the primary driversshould be investigated in the context of a particularportfolio.

    Port fo l io Parameters

    (Base Scenario)

    Number of loans 6 000

    Total exposure EUR1.0bnRange (%)

    PD PD < 0.2% 380.2% PD 0.7% 470.7% < PD 1.0% 15

    LGD (%) Exposure weighted: 36Correlation Basel estimates:

    0.12 0.24Grades 15

    Source: Fitch

    Base Scenar io

    The distribution of exposures (as a percentage ofEUR1bn total exposure) per PD band is shown in thefigure below for the portfolio.

    0

    2

    4

    6

    8

    10

    12

    0.0 0.2 0.4 0.6 0.8 1.0

    PD (%)

    (Exposure %)

    Portfolio Value

    Source: Fitch

    PD Bands & Exposur e Distr ibut ion

    Portfolio

    PD band Average (%) % of total exposure1 0.06 4.642 0.13 5.733 0.15 6.154 0.18 10.245 0.20 10.856 0.23 10.707 0.44 9.548 0.49 7.399 0.54 7.2810 0.63 6.65

    11 0.68 5.5712 0.84 4.8813 0.89 4.6714 0.95 3.9015 0.98 1.81

    Source: Fitch

    Scenario TestsTest Description

    1 8 PD grades: The average and standard deviationfor each PD band are therefore largerthan in thebase scenario.

    2 22 PD grades: The average and standard deviationfor each PD band are therefore smallerthan in thebase scenario.

    3 PDs all up 20%. Standard deviations of PD bandsremain the same as the base scenario.

    4 PDs all down 20%. Standard deviations of PD bandsremain the same as the base scenario.

    5 Exposure weighted LGDs all up 20%.6 Exposure weighted LGDs all down 20%.7 Correlation up 20%

    8 Correlation down 20%Source: Fitch

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    8/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    8

    Percentage of portfolio exposure by PD band.The table below shows the average PD and the asso-ciated standard deviation of each PD band for theportfolio.

    For the base scenario, the results and capital chargeswere as follows:

    Summary of Measures Base

    Scenario

    Total exposure EUR1,000,000,000

    Expected loss EUR1,436,692Unexpected loss @ 99/9% EUR53,355,699Capital charge 5.34%

    Source: Fitch

    The Impact o f Change in PD

    0

    400,000

    800,000

    1,200,000

    1,600,000

    2,000,000

    8 grades 15 grades 22 grades

    Different PD Bands: Basic Model EL

    Source: Fitch

    (EUR)

    The portfolio was tested at different threshold levelsfor all PD bands. Higher average PDs increase theUL considerably. This result is not unexpected:lower average PDs result in larger thresholds whichmust be breached before default (and hence a loss)can occur. Since these thresholds will be more diffi-cult to breach than those for higher average PDs,

    losses are reduced.

    If starting with more than 15, increasing the numberof PD buckets used to allocate input data does nothave a significant effect on the EL, and the valuesconverge to the base case EL. However, a decreasingnumber of PD buckets does result in portfolio ELincreasing by 20%. Note that if the net exposureweighting method were used here, the portfoliowould be mean invariant, so that the number ofbuckets would have no effect on EL.

    0

    10,000,000

    20,000,000

    30,000,000

    40,000,000

    50,000,000

    60,000,000

    70,000,000

    8 grades 15 grades 22 grades

    Different PD Bands:

    Basic Model UL @ 99.9%

    Source: Fitch

    (EUR)

    The UL is more sensitive than the EL to the numberof PD buckets and increases by 22% when PD buck-ets are restricted to eight grades. Conversely, whenPD buckets increase to 22 grades UL decreases by10%. When assessing lower-quality portfolios orincluding migration analysis, the impact of ratingscale granularity must be explored in greater depth.

    0

    10,000,000

    20,000,000

    30,000,000

    40,000,000

    50,000,000

    60,000,000

    PD 20% up from

    base

    PD at base level PD 20% down

    from base

    Average PD Changes: Basic

    Model UL @ 99.9%

    Source: Fitch

    (EUR)

    Increasing the average PD of the PD bands increasesthe UL by 7%. Decreasing the average PD of the PDbands decreases the UL by 17%. This result is sensi-tive to the interaction between the overall averagePD level (i.e., credit quality) and the confidencelevel.

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    9/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    9

    0

    10,000,000

    20,000,000

    30,000,000

    40,000,000

    50,000,000

    60,000,000

    Basel threshold PDs Fitch threshold PDs

    Different PD Thresholds: Basic Model v.

    Basel II: UL @ 99.9%

    Source: Fitch

    (EUR)

    The Impact o f Chang ing LGD

    0

    10,000,000

    20,000,000

    30,000,000

    40,000,000

    50,000,000

    60,000,000

    70,000,000

    LGD 20% up

    from base

    LGD at base

    level

    LGD 20% down

    from base

    LGD Changes: Basic Model UL @ 99.9%

    Source: Fitch

    (EUR)

    Increasing the exposure weighted LGDs increasesthe UL by 13%, while reducing the LGDs decreasesthe UL by 6%. Note that the in this example, UL isless sensitive to changes in LGD than average PDs.This is a common property of models, and directconsequence of the construction of the model, but itcan be influenced by assumptions relating to stochas-

    tic LGD.

    The Impact o f Chang ing Cor re la t ion

    Increasing the correlation by 20% increased the ULby 10%, while decreasing the correlation decreasedthe UL by 12%.

    In assessing this result, note that a single-factormodel assuming infinite granularity has been used tocompute results at the 99.9% level. Using a multi-factor model, modelling concentrations, and/orchanging the confidence level (esp. to 99.97%) candramatically increase the impact of correlations on

    the results. Typically, if concentrations are modelledin portfolios of 6000 large corporate loans (as in this

    example), correlation is the dominant driver of re-sults in the tail.

    0

    10,000,000

    20,000,000

    30,000,000

    40,000,000

    50,000,000

    60,000,000

    70,000,000

    Correlations

    20% up

    Basel calculated

    correlations

    Correlations

    20% down

    Correlation Changes:

    Basic Model UL @ 99.9%

    Source: Fitch

    (EUR)

    Capi ta l isa t ion Leve ls

    The results in all of the above scenarios were for abank capitalised at a AA level ie a confidencelevel of 99.9% (or a probability of default of 0.1%).For illustrative purposes the effect on the UL at vari-ous confidence levels and PDs linked to ratings isshown below. It should be noted that institutionsmay well choose to link their calculations to differ-ent parameter values which are equally appropriate

    Summ ary o f cap i ta l c harges

    Rating Bank capitalised atCapital Charge as

    a % of exposure

    AAA (PD = 0.03%, CI = 99.97%) 7.31AA (PD = 0.10%, CI = 99.90%) 5.34A (PD = 0.40%, CI = 99.60%) 3.35BBB (PD = 1.00%, CI = 99.00%) 2.31BB (PD = 2.00%, CI = 98.00%) 1.58B (PD = 4.00%, CI = 96.00%) 1.04CCC (PD = 8.00%, CI = 92.00%) 0.58

    Source: Fitch

    Economic Cap ita l vs . Regu la tory Cap i ta l

    Regulatory Capital: Regulatory capital focuses ondifferent tiers of capital and relates the level ofcapital to risk-adjusted assets. Risk is incorporatedinto the calculation of capital by the assignment ofassets into simple discrete risk buckets. Currentlyrisk-weightings are assigned by the regulators andare driven primarily by their view of the amount ofcapital required within the banking system to protectthe stability and soundness of the financial system.Unfortunately, the existing regulatory approach tocapital is not risk-sensitive and over time was in-creasingly seen to be divorced from the underlyingrisks borne by an institution. This is due to change

    with the implementation of Basel II, which seeks toalign regulatory capital to risk more closely.

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    10/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    10

    Economic Capital: By contrast, from a conceptualpoint-of-view economic capital is a measure of risknot of capital held. In practice, in most institutions itis considered to be a buffer of capital to guard

    against unexpected loan losses arising from credit,operational, market and business risks over a one-year horizon and is calculated at a pre-specified con-fidence level. Fitch believes it is a more forwardlooking requirement as it is based on an assessmentof potential future losses focussing on changes in thetrue economic value of assets (not earnings). Eco-nomic capital should provide an indication of theflexibility the bank needs in planning for growth.

    Comparison of the Basel-calculated capital chargeswith those obtained from the Basic Model (also asingle-factor credit risk model) showed some differ-

    ence. However, the results from the Basel Model arealso considerably affected by the number of PDgrades (with more PD grades giving more consistentresults). Recalling the gross-exposure weighted PDs,this result is expected. Other observations are possi-ble with net-exposure methods.

    25,000,000

    30,000,000

    35,000,000

    40,000,000

    45,000,000

    50,000,000

    55,000,000

    Basic Model Basel Model

    Basic Model v. Basel II: UL @ 99.9%

    Source: Fitch

    (EUR)

    The UL calculated using the Basic Model and theBasel Model exhibit differences which stem form thefact that the Basel equations are closed form andincorporate an intrinsic maturity factor. Model re-

    sults however, converge for increasing PD gradegranularity. EL and UL values increase with increas-ing PDs, LGDs and correlations in both models.

    From Fitchs perspective, the overall aim is clearlyto base capital requirements on a true economic capi-tal assessment. However, reliance on a banks inter-nal economic capital model requires confidence thatit produces capital levels commensurate with therisks being run. This is not a straightforward task,and realistically both economic and regulatory capi-tal requirements will need to be considered in Fitchsanalysis of a banks capital adequacy.

    Appendix 2 provides more detail concerning thequestions Fitch expects to be asking Banks duringthe review process.

    The answers to these questions will provide us withan ability to develop a detailed understanding andlevel of comfort with a credit risk model. We alsoexpect institutions to perform certain stress tests tokey assumptions and provide detailed reports of theresults. These reports combined with the model out-puts of a sample portfolio from a multitude of ratedinstitutions will enable us to fully understand themodels employed and contrast them against bench-marks.

    Credit risk modelling continues to evolve, and a lackof practical back-testing alternatives means that the

    jury is still out regarding their effectiveness overtime. Therefore, Fitch's opinion stresses that cautionmust be exercised when granting credit to credit riskmodels.

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    11/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    11

    Appendix 1

    Credit-Worthiness Model l ing Approaches

    The methodologies underlying contemporary credit

    risk measurement models can be traced to two dif-ferent approaches to asset pricing: a structural ap-proach pioneered by Merton in 1974 and a reduced-form approach employing intensity-based models toestimate stochastic hazard rates launched by Jarrowand Turnbull in 1995, further developed by Jarrow,Lando, and Turnbull in 1997 and then by Duffie andSingleton in 1998/1999.

    Whilst these two methodologies differ somewhat intheir approach, both tackle a central theme commonto all credit risk measurement models: the estimationof default probabilities. Commercially availablemodels use at least one of the methodologies referredto above. A discussion of the different approaches ispresented below as background.

    Struc tura l Approaches

    The structural approach to credit default modelstreats a credit-risky bond (or swap) as a credit risk-less bond (or swap) minus an option to exchange thebond (or swap) for the debtors entire portfolio in theevent of bankruptcy. Merton modelled a leveredfirms equity as a call option on the firms assetswith a strike price equal to the debt repaymentamount. If at expiration (coinciding with the matur-ity of the firms liabilities, assumed to be comprisedof pure discount debt instruments) the market valueof the firms assets exceeds the value of its debt, thefirms shareholders will exercise the option to re-purchase the companys assets by repaying the debt.However, if the market value of the firms assetsfalls below the value of its debt, the option expiresworthless and the firms shareholders default.

    One common model estimates the probability of de-fault1, the Expected Default Frequency (EDF), foreach obligor based on a Merton-type model. Theprobability of default is thus a function of the firm'scapital structure and the credit risk driven by the

    dynamics of the issuer's asset value (ie the volatilityof the asset returns and the current asset value).Given the firm's current capital structure (ie thecomposition of its liabilities: equity, short-term andlong-term debt, convertible bonds, etc.) and havingspecified the stochastic process which drives theasset value, the probability of default for any timehorizon, (one year, two years, etc.) may be calcu-lated. Typically, the EDF is firm-specific and can bemapped into any rating system to derive the equiva-lent rating of the obligor. EDFs can be viewed as a

    1

    This model does not make any explicit reference to transition probabilities, since theseare already imbedded in the EDFs. Instead, each value of the EDF is associated with aspread curve and an implied credit rating.

    cardinal ranking of obligors relative to default risk,instead of the more conventional ordinal rankingused by rating agencies.

    This type of model best applies to publicly traded

    companies for which the value of equity is market-determined. While rooted in a sound theoretical basis,it is only practical for firms with simple capital struc-tures; it is questionable for those with capital struc-tures that are more complex such as financial institu-tions and international conglomerates.

    In addition, Mertons structural model issues earlywarnings of default by allowing a gradual diffusionof asset values to the default point (equal to the debtlevel). As a result, this process implies that the PDsteadily approaches zero as the time to maturity de-creases a result not observed empirically in the

    credit spread term structures.

    Reduced form Approaches

    In contrast to Merton models, credit spreads that aremore firmly rooted in practical experience are ob-tained from reduced form or intensity-basedmodels.Hence, whilst Merton's structural approach modelsdefault as the result of a gradual deterioration in as-set values, intensity-based approaches model defaultas sudden, unexpected events.2 These models esti-mate PDs that are more consistent with empiricalobservations even though they do not specify theeconomic process leading to default. Instead, model

    default is a point process in which defaults occurrandomly with a probability determined by the inten-sity or hazard rate. Intensity-based models decom-pose observed credit spreads on defaultable debt, orCredit Default Swaps (CDS), to determine the PD(conditional on no default having occurred prior to a

    specified time) as well as the LGD (given by R1 ,

    where R is the recovery rate).

    Intensity-based models are essentially empirical,employing (observable) risky debt prices and creditspreads to measure the stochastic jump processwhich governs default.

    Whilst theoretically pleasing, the model is not with-out its share of practical problems. The fundamentalproblem exists that calibration of reduced form mod-els requires the estimation of two variables at once:PD and LGD. So observing merely one creditspread is insufficient. Different workarounds havebeen tried such as observing a term structure ofspreads for the firm and assuming one LGD for thewhole or pegging PD using the equity price, and thenpegging LGD using debt instruments of differingseniority.

    2

    Reduced-form models decompose risky debt prices in order to estimate the randomintensity process underlying default, whereas the structural approach models the eco-nomic process of default.

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    12/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    12

    Appendix 2 - Possib le Quest ions

    In order to assess an institution's credit risk model,the starting point for Fitch is a clear understanding ofthe principles and techniques that the model uses. Inorder to build up this information, Fitch seeks infor-mation on:

    High-Level Overview

    Is the credit risk model in use a standard vendordeveloped model, an in-house model or a com-bination of both?

    What were the prime factors in determining thevendor credit risk model or the in-house modelin use?

    What are the principal assumptions and settingsof the model in use (one factor, multi-factor, de-fault definition, mark-to-model)?

    Are the parameters and assumptions appropriateto the institutions portfolio and business mod-els?

    What software tools are used for modelling?How many model assumptions are dictated bythe tool?

    What is the probability of survival and time ho-rizon used (eg 99.90% over one year or anyother confidence level) by the credit risk model?Does this change with the decision being sup-ported? Where applicable, how does this differfrom your current rating (e.g. AA: 99.95%) andthe regulatory standards?

    Where appropriate, is the modelling sophistica-tion commensurate with the portfolio holdings?

    Please explain how the outputs from the modelfeed into the risk management and decision-making process of the bank.

    Model Deve lopment

    Is the vendor credit risk model in use tailored tomeet your in-house requirements or is it imple-mented with no change?

    If using an in-house model or a tailored vendormodel, was model development and changes

    done in-house or externally by consultants? Or acombination of the two?

    If made or customized in-house, was this drivenby a profit centre of the institution or a central-ized risk management department?

    Where applicable, is there a separate model forcorporate, financial institutions, sovereigns, pro-

    ject finance, mortgages, revolving retail loansand other retail loans?

    Where applicable, do the models differ by coun-try of operation or are they the same acrosscountries?

    What testing was performed on the model prior

    to implementation? And as ongoing validation?

    Has the model been validated or reviewed byany external agency (eg regulators, consultants)?

    What are the main areas for ongoing and futuredevelopment?

    Inputs o f the Model

    Fitch does not expect to see every possible feature aspart of every model but expects complexity to becommensurate with that of the organisation.

    PD

    How are PDs quantified for the following assetclasses: corporate, financial institutions, sover-eigns, mortgages, revolving retail and other re-tail loans. (eg historical default experience, sta-tistical model and external mapping approach)?

    How do you set a PD to represent a pool ofmany similar (homogeneous) obligors (eg retailloans, credit card loans)?

    Can you elaborate on the number of risk buckets(i.e., credit rating grades) used by asset class andthe rationale for using these risk buckets in con-trast to a higher or lower number of risk buckets(e.g. 8 vs 15 vs 22)?

    Can you also explain:- the number of obligors per risk rating bucket?- the difference if any between expected and re-

    alised PD?- the mean and the standard deviation of PDs byrisk bucket?

    If your PDs are stressed PDs, explain the differ-ence with unstressed PDs by asset class. If PDsare unstressed, then explain whether they are atall stressed and if so how you factor in stressedPDs.

    What definition of default do you use for PDestimation and loss evaluation? How different isthis from the accounting definition of default,

    and do you reconcile these differences?

    Where applicable, if the model takes into ac-count rating migration, can you provide the de-fault rates observed per rating grade?

    LGD

    What is the approach adopted to quantifyingLGD for the following asset classes: corporate,financial institutions, sovereigns, mortgages, re-volving retail and other retail loans. (eg expert

    judgement, implied market LGD or discountedcash-flow approach)

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    13/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    13

    Do you determine LGD values with the help ofa grading system? Can you explain in brief howthis system works? Where you do not use agrading system, explain how you capture LGD.

    Does the model take into account a stressedLGD for all the asset classes?

    For low default portfolios, how do you estimateLGD values?

    Do you benchmark your LGD values?

    Provide the definition of default used (in inter-nal grading?) and if it is not consistent with thedefinition used in determining PDs, an explana-tion as to why.

    In estimating LGD values, do you account forrecoveries and costs? Explain briefly how.

    Are collateral values adjusted in arriving at theLGD estimates? Explain briefly what type ofcollateral and the valuation process for the col-lateral (eg guarantees, mortgages and receiv-ables).

    Where applicable, if using workout LGDmethod to calculate LGD values, can you ex-plain the discount rates used?

    If there are significant differences between real-ised LGD and LGD estimates, can you explainthem?

    EA D

    How do you incorporate drawn amounts and anestimate of future draw-downs in determiningEAD? Explain the method used in estimating fu-ture draw-downs.

    Where un-drawn limits are treated as uncondi-tionally cancellable, explain the rationale for do-ing so (eg credit card limits, interbank lines).

    Valuat ion Methodo logy

    How is valuation performed? At the loan hori-zon, or are loan values present values?

    How are coupon frequencies taken into account?

    How is the trading book valued, and why?

    Matu r i t y

    Explain how maturity is incorporated in themodel for loans with maturity greater than oneyear (i.e. refers mainly to mortgage and other re-tail loans).

    Explain how maturity is incorporated in themodel for loans with maturity less than one year.

    Where maturity is greater than one year, explainhow defaults beyond one year are addressed.

    Corre la t ion

    What correlation is measured: asset, default,equity?

    If equity correlation, how is this translated intoasset correlation, and from there into defaultcorrelation?

    If a single-factor model is used, which economicfactor is considered?

    If a multifactor model is used, which factors areconsidered and how are the correlations ob-tained?

    How is the level of idiosyncratic risk determined

    for each obligor or class of obligors? What granularity assumptions are made? (Esp.

    any differences as regards retail, SME and cor-porates)

    Model Output

    What are the key outputs of the model (capitalrequired at what confidence level, capital bybusiness line)?

    What are the primary uses of model outputs?Include both direct and indirect uses (regulators,rating agencies, pricing, performance measure-ment and others).

    How is diversification taken into account by themodel? (eg between risk types and entities)

    What methodology is used to allocate risk backto subsets of the business?

    Model Use

    How often is the model used?

    How many iterations does the model employ forthe various asset classes?

    If the overall bank credit risk charge is derivedfrom several sub-models, describe how theoverall results are aggregated.

    Do the sub-models differ by country?

    Model Va l ida t ion

    Validation is a key issue in the use of rating systems.The development of models to effectively and ade-quately assess obligors credit risk is severely ham-pered by limited availability of objective model in-puts. Most models in current use measure creditwor-thiness over a single year or more, but these modelsrequire several years of historical financial data foreach borrower.

    Large corporate borrowers are replete with reliable

    and timely financial data. Data for smaller borrowersis uncommon and often unreliable, even less so for

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    14/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    14

    those corporates in financial distress (which are key,after all, to the construction of accurate credit riskmodels). The highly sporadic nature of defaultevents also results in a scarcity of reliable data re-

    quired for building credit risk models.

    This paucity of data presents challenges in assessingthe accuracy and reliability of credit risk models.The Basel Committee has also highlighted the rela-tively informal nature of credit model validation ap-proaches at many financial institutions in its recentcredit risk modelling report. In particular, the Com-mittee has emphasised both data sufficiency andmodel sensitivity analysis as significant challengesto the validation process.

    For these reasons, it is critical that all inputs to themodel, and the main model assumptions be scruti-nized in detail. For example, one must assess thevalidity of the PD models in use. The following sec-tion provides an illustration of validation of a riskrating system.

    Val ida t ing a Risk Rat ings System : Input PD

    Default risk models, as classification tools, can err inone of two ways. Type I errors result from modelswhich indicate low risk when the risk is high andthese correspond to the assignment of high creditquality to issuers who actually do default or ap-proach default. Investors may experience losses onprincipal and interest, or in the market value of theobligation. Type II errors result from those modelsassigning low credit quality when the credit qualityis high. Losses resulting from these errors includethe loss of both return and origination fees whenloans are lost by either lender rejection or non-competitive bidding. Minimising one of these errors,however, increases the other. This trade-off givesrise to complex and important issues which are ad-dressed in the validation process.

    Validation refers to all activities undertaken to ana-lyse and verify the overall performance of the riskrating system in order to ensure both consistency andaccuracy. In general, the validation procedure entailsthe following activities:

    Confirmation of the conceptual soundness andinitial risk quantification of the risk rating sys-tems design, including the concept, methodol-ogy and assumptions

    Confirmation of the operation of the risk ratingsystems, including the:

    discriminatory power of the risk rating sys-tem;

    calibration of the risk rating system; and the

    risk homogeneity of the obligors in each ofthe risk pools.

    Regular examination of overall performance of

    the risk rating system including back-testing (ieperformance assessment of a risk rating system(including PDs, LGDs and EADs) based on his-torical data. This examination will also comparerealised and predicted outcomes and bench-marking (ie the comparison of the performanceof two different, but comparable, risk rating sys-tems).

    Back-testing measures the historical per-formance relative to some other benchmarkmodel, hence back-testing is merelybenchmarking of historical performance.

    Realised default and migration rates mustbe checked to ascertain whether they areconsistent and are supported by historicalevidence from internal data. Systematicallyhigh or low PDs must nevertheless maintainthe ordinal ranking of the obligors (ie therelative riskiness must be preserved) andconsequently the calibration of the PD as-signed and the actual observed number ofdefaults requires a quantitative, consistencytest.

    Rigorous statistical testing is required be-

    fore any conclusions may be drawn aboutwhether assigned PDs are appropriate or not.

    Performance statistics for risk rating systems can behighly sensitive to the data sample used for valida-tion. Quantitative models should ultimately aim toavoid this unwanted sample dependency. The bank-ing industry is currently inundated with statisticaltests which gauge and rank various aspects of creditmodel performance. This is the subject of a futureFitch publication and will not be covered exhaus-tively here. Two simple measures which aim to cap-ture overall model performance or accuracy are the

    Cumulative Accuracy Profile and the2

    -test.

    Cumula t ive Ac curacy Prof i les (CAPs)

    Cumulative Accuracy Profiles or CAPs (also calledreceiver-operator curves and power curves) allow avisual qualitative assessment to be made of modelperformance. CAPs refer specifically to cases inwhich the curve represents the cumulative probabil-ity of default over an entire population, ie not onlythe non-defaulting population.

    If a model assigns random risk scores, the fraction of

    defaulters will be roughly the same as the fraction oftotal companies, thereby generating a straight line or

  • 8/8/2019 How Much Credit Crdt Rsk Mdls

    15/15

    Banks

    How much Credit in Credit Risk Models?: May 2007

    15

    Random CAP. A perfect model will produce theIdeal CAP a straight line capturing 100% of thedefaults within a fraction of the population equal tothe fraction of defaulters in the sample. Because the

    fraction of defaulters is usually a small number, theIdeal CAP is steep, as shown in the figure overleaf.A useful property of CAPs is that they reveal infor-mation about the discriminatory power of the modelover its entire range of risk scores for a particulartime horizon.

    2

    - test

    The2

    -test is a statistical test, applied to and inter-

    preted using the CAP curve. Under a simple statisti-cal test approach, the null hypothesis is that ratingsare no better than random chance. If this were true,

    then the number of defaults in each rating gradewould be the same percentage as for the entire sam-

    ple. The2

    -test algorithm calculates the expected

    defaults per grade and compares them with actualdefaults per grade (by tallying up the individual de-faults by grade and total obligors by grade). The

    2

    -probability test measures the statistical signifi-

    cance of the2

    - statistic being better than random

    chance.

    References

    1. Basel Committee on Banking Supervision,Core Principles Methodology, (2006), Bankfor International Settlement.

    2. Basel Committee on Banking Supervision,Sound credit risk assessment and valuation forloans, (2006), Bank for International Settle-ment. [link]

    3. Basel Committee on Banking Supervision, In-ternational Convergence of Capital Measure-

    ment and Capital Standards - A Revised

    Framework, (2005), Bank for InternationalSettlement.

    4. Engelmann, B., E. Hayden, D. Tasche, (2003):Testing Rating Accuracy, RISK, Vol. 16, No1, pp 82-86.

    5. Tasche, Dirk, Validation of Internal RatingSystems and PD Estimates, (2006), workingpaper, Deutsche Bundesbank.

    6. Rutter Associates LLC, Convergence of CreditCapital Models, (2006).http://www.isda.org/c_and_a/pdf/IACPM-ISDA-ConvergenceCreditCapModelsFeb2006.pdf

    Copyright 2007 by Fitch, Inc., Fitch Ratings Ltd. and its subsidiaries. One State Street Plaza, NY, NY 10004.Telephone: 1-800-753-4824, (212) 908-0500. Fax: (212) 480-4435. Reproduction or retransmission in whole or in part is prohibited except by permission. All rights reserved. All of theinformation contained herein is based on information obtained from issuers, other obligors, underwriters, and other sources which Fitch believes to be reliable. Fitch does not audit orverify the truth or accuracy of any such information. As a result, the information in this report is provided as is without any representation or warranty of any kind. A Fitch rating is anopinion as to the creditworthiness of a security. The rating does not address the risk of loss due to risks other than credit risk, unless such risk is specifically mentioned. Fitch is notengaged in the offer or sale of any security. A report providing a Fitch rating is neither a prospectus nor a substitute for the information assembled, verified and presented to investors bythe issuer and its agents in connection with the sale of the securities. Ratings may be changed, suspended, or withdrawn at anytime for any reason in the sole discretion of Fitch. Fitch doesnot provide investment advice of any sort. Ratings are not a recommendation to buy, sell, or hold any security. Ratings do not comment on the adequacy of market price, the suitability ofany security for a particular investor, or the tax-exempt nature or taxability of payments made in respect to any security. Fitch receives fees from issuers, insurers, guarantors, otherobligors, and underwriters for rating securities. Such fees generally vary from US$1,000 to US$750,000 (or the applicable currency equivalent) per issue. In certain cases, Fitch will rateall or a number of issues issued by a particular issuer, or insured or guaranteed by a particular insurer or guarantor, for a single annual fee. Such fees are expected to vary from US$10,000to US$1,500,000 (or the applicable currency equivalent). The assignment, publication, or dissemination of a rating by Fitch shall not constitute a consent by Fitch to use its name as anexpert in connection with any registration statement filed under the United States securities laws, the Financial Services and Markets Act of 2000 of Great Britain, or the securities laws ofany particular jurisdiction. Due to the relative efficiency of electronic publishing and distribution, Fitch research may be available to electronic subscribers up to three days earlier than toprint subscribers.