Social Security: A-02-05-15116

8/14/2019 Social Security: A-02-05-15116

http://slidepdf.com/reader/full/social-security-a-02-05-15116 1/27



– The Commissioner

Please provide within 60 days a corrective action plan that addresses eachrecommendation. If you wish to discuss the final report, please call me or have yourstaff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at(410) 965-9700.

SPatrick P. O’Carroll, Jr.

Attachment



OFFICE OF

THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:

OUTSTANDING DEBT

January 2006 A-02-05-15116

AUDIT REPORT



Mission

We improve SSA programs and operations and protect them against fraud, waste,

and abuse by conducting independent and objective audits, evaluations, andinvestigations. We provide timely, useful, and reliable information and advice toAdministration officials, the Congress, and the public.

Authority

The Inspector General Act created independent audit and investigative units,called the Office of Inspector General (OIG). The mission of the OIG, as spelledout in the Act, is to:

Conduct and supervise independent and objective audits and

investigations relating to agency programs and operations. Promote economy, effectiveness, and efficiency within the agency. Prevent and detect fraud, waste, and abuse in agency programs and

operations. Review and make recommendations regarding existing and proposed

legislation and regulations relating to agency programs and operations. Keep the agency head and the Congress fully and currently informed of

problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform. Access to all information necessary for the reviews. Authority to publish findings and recommendations based on the reviews.

Vision

By conducting independent and objective audits, investigations, and evaluations,we are agents of positive change striving for continuous improvement in theSocial Security Administration's programs, operations, and management and inour own office.



MEMORANDUM

Date: January 23, 2006

To: Inspector General

From: PricewaterhouseCoopers, LLP

Subject: Performance Indicator Audit: Outstanding Debt (A-02-05-15116)

OBJECTIVE

The Government Performance and Results Act (GPRA)1 of 1993 requires the SocialSecurity Administration (SSA) to develop performance indicators that assess therelevant service levels and outcomes of each program activity.2 GPRA also calls for a

description of the means employed to verify and validate the measured values used toreport on program performance.3

Our audit was conducted in accordance with generally accepted government auditingstandards for performance audits. For the performance indicator included in this audit,our objectives were to:

1. Assess the effectiveness of internal controls and test critical controls over thedata generation, calculation, and reporting processes for the specificperformance indicator.

2. Assess the overall reliability of the performance indicator’s computerprocessed data. Data are reliable when they are complete, accurate,consistent and are not subject to inappropriate alteration.4

3. Test the accuracy of results presented and disclosed in the Fiscal Year (FY)2004 Performance and Accountability Report (PAR).

4. Assess if the performance indicator provides a meaningful measurement ofthe program it measures and the achievement of its stated objective.

1Public Law Number 103-62, 107 Stat. 285 (codified as amended in scattered sections of 5 United States

Code (U.S.C.), 31 U.S.C. and 39 U.S.C.).

231 U.S.C. § 1115(a)(4).

3 31 U.S.C. § 1115(a)(6).

4GAO-03-273G, Assessing Reliability of Computer Processed Data , October 2002, p. 3.

Performance Indicator Audit: Outstanding Debt (A-02-05-15116) 1



BACKGROUND

We audited the following performance indicators as stated in the SSA FY 2004 PAR:

Performance Indicator FY 2004 Goal FY 2004 Reported Results

Percent of Outstanding OldAge, Survivors, andDisability Insurance(OASDI) Debt in aCollection Arrangement

38% 42.2%

Percent of OutstandingSupplemental SecurityIncome (SSI) Debt in aCollection Arrangement

55% 53.5% (May – September)5

Under the OASDI and SSI programs, SSA issues benefit payments to millions ofbeneficiaries for various entitlement programs. At times, SSA pays beneficiaries morethan they are entitled to receive. This results in an overpayment that SSA thenrecognizes as an outstanding debt owed by the beneficiary. During the audit period,SSA had $4 billion dollars of OASDI and SSI debt that had not been resolved (i.e. hasnot been collected, removed as not actually an overpayment, waived, or written offconditionally). New debt that is determined to be an outstanding overpayment eligiblefor collection primarily occurs as result of errors in: Wages/Self-Employment Income(Old Age, and Survivors Insurance (OASI) and SSI), Computations (OASI), AnnualEarnings Test (OASI), Relationship/Dependency (OASI), Substantial Gainful Activity(DI), Unreported Death (DI), Workers’ Compensation (DI) and Financial Accounts (SSI).

SSA classifies outstanding debt in four principal categories: 1) newly established debt(i.e. debt in a due process period and less than 60 days old), 6 2) debt under appeal(i.e. debt with an appeal pending and less than 90 days old), 3) debt in a collectionarrangement (i.e. collection is anticipated or debt is being collected, primarily throughtimely installments or monthly benefit withholding), and 4) debt not in a collectionarrangement (i.e. all remaining debt that is neither in collection nor anticipated to becollected, but requires further action). These performance indicators were developed totarget an average annual percentage of debt in a collection arrangement compared tothe overall status of outstanding debt.

When an overpayment is detected, it is initially placed in the "newly established debt"

category and may remain in this category for up to 60 days, the maximum periodpermitted for a beneficiary to appeal or apply for a waiver. After 60 days, the debtmoves to either “debt in a collection arrangement” or “debt not in a collection

5SSA’s Office of Quality Assurance and Performance Assessment (OQA) determined that the data from

October 2003 – April 2004 was incorrect due to a system error that resulted in an inaccurate calculationduring this period. As a result, the data reported was for the period May 2004 - September 2004.

6Debt in this category can be appealed at any time during the 60-day period.




arrangement." At any time, if the overpayment has been determined to be “debt in acollection arrangement,” the amount would be moved to this category in the monthlydebt report, which is used to calculate the indicator results. Delinquent debt, i.e. “debtnot in a collection arrangement” is typically referred to the Treasury Offset Program andcredit bureaus (i.e. Equifax, TransUnion, Innovis and Experian).7

At the end of each month, SSA uses software packages (“DEBTCATC” for OASDI and“SSIDEBT29” for SSI) to review the data generated from the relevant source systems(Recovery of Overpayment, Accounting and Reporting (ROAR) System for OASDI andthe Supplemental Security Record (SSR) for SSI). The software packages determinethe status of each unresolved debt based on the four principal categories and then sumthe amounts for each principal category to determine the proportion of debt in acollection arrangement. The percentage calculated represents the proportion of debt ina collection arrangement for the most recent 12 months. The indicator resultspresented and disclosed in the PAR represent a 12-month rolling average.

Performance Indicators’ Calculation

% of Debt in a CollectionArrangement

=Total Dollar Amount in a Collection

Arrangement / Total Dollar Amount ofOutstanding Debt

(For additional detail on the calculation of these indicators, refer to the flowcharts inAppendix C.)

These performance indicators are linked to the SSA strategic objective to “Prevent

fraudulent and erroneous payments and improve debt management,”8

which is linked tothe SSA strategic goal “To ensure superior Stewardship of Social Security programsand resources.” 9 This strategic goal is linked to one of the five Governmentwide goalson the President’s Management Agenda, “Improved Financial Management,”10 whichdirects agencies to set goals to reduce improper payments for each of their programs.11 To assess its progress in meeting this objective, SSA established these performanceindicators to measure the percentage of OASDI and SSI debt in a collectionarrangement.

7Written-off debt, which includes some delinquent debt, remains eligible for collection and benefit offset,

should the individual return to the benefit rolls; however, this debt is no longer included in the

performance indicators’ calculation. 8

Social Security Administration Performance and Accountability Report Fiscal Year 2004, p. 85.

9 Social Security Administration Performance and Accountability Report Fiscal Year 2004, p. 85.

10 http://www.whitehouse.gov/omb/budget/fy2002/mgmt.pdf.

11 Social Security Administration Performance and Accountability Report Fiscal Year 2004, p. 27 statesthat SSA uses the terminology “improper payments” and “erroneous payments” interchangeably.




RESULTS OF REVIEW

We found SSA did not have adequate internal controls over the processes used togenerate, calculate, and report the performance data used in the performance indicators

and did not retain support for the calculations.12

As a result, we were unable toexamine sufficient support for the reported indicators to conclude on the accuracy of thecalculation. Additionally, the indicators did not provide a clear linkage of: SSA’sprogress to align with existing Federal debt collection laws; relationships tocomplementary debt collection measures to enable a comprehensive assessment ofoverall debt collection efforts; and SSA’s decision to maintain, not raise, annualperformance targets. Lastly, SSA can further develop a more meaningful indicator tomeasure debt that is eligible to be collected.

Internal Controls

We found that SSA did not have sufficient documentation of the indicator’s intendedpurpose, automated and manual controls in operation, accountability for indicatormeasurement, a comprehensive set of supporting data sources used in the calculation,and step-by-step procedures to reperform the indicator results from origination topublication in the PAR.13 Accordingly, SSA did not comply with standards defined bythe Office of Management and Budget’s (OMB) Circular A-123, Management Accountability and Control that require, “The documentation for transactions andmanagement controls, and other significant events must be clear and readily availablefor examination.”14

The following finding is specific to the “Percent of Supplemental Security Income (SSI)

Debt in a Collection Arrangement” performance indicator. During our FY 2004 FinancialStatement Audit, we found that SSA policy was not adhered to for approvals ofoverpayment waivers over $2,000 in 1 out of 9 field offices examined. Further, wefound inaccurate overpayment calculations existed in 1 out of 46 overpaymentsexamined. In addition, during an audit of SSA’s Controls over the Title XVIOverpayment Waiver Process,15 the SSA Office of the Inspector General foundsignificant deficiencies in documentation support and policy decisions for waiver

12 This statement applies to the performance indicators included in this report and is not intended to applydirectly to SSA’s internal controls over financial reporting.

13We held a meeting with the Agency on March 24, 2004, to discuss the Results of Review for these

performance indicators. Subsequent to our meeting, the Agency drafted new documentation to define thepolicies and procedures in place during the audit period. We informed the Agency that we wouldacknowledge its efforts to improve the indicators’ documentation; however, we were unable to verify ortest the completeness and accuracy of this new documentation as it was not in place during the period ofour audit.

14 Revision to OMB Circular A-123, Management Accountability and Control , June 21, 1995.

15 Social Security Administration’s Controls over the Title XVI Overpayment Waiver Process (A-06-03-13077) issued October 25, 2004.




approvals. These weaknesses identified in internal controls over overpaymentprocessing may impact the indicator results, as the overpayments waived would haveaffected the denominator for the calculation of this indicator, and thereby incorrectlyincrease the percentage reported.

Data Reliability and Accuracy of PAR Presentation and Disclosure

We found that SSA did not archive the data supporting the performance indicators’calculation. As a result, we were unable to conclude on the reliability of the computerprocessed data used to support the performance indicators’ calculation. In addition, asa result of our inability to recalculate the performance results, we were unable todetermine if the results reported in the PAR were accurately presented and disclosed.SSA is currently examining data storage possibilities to address this issue.

We found that SSA’s programmers have ALL access to production datasets. This levelof access would allow users to create, delete and update any of the data we reviewed

without appropriate review or approval of the changes. This access level does notcomply with generally accepted standards for limited access control defined by OMBCircular A-130, National Institute of Standards and Technology (NIST) 800-12, andNIST 800-14. SSA is currently working to address the risks associated with this findingthrough the Standardized Security Profile Project. These datasets should be securedover the next 3 years.

The following finding is specific to the “Percent of Supplemental Security Income (SSI)Debt in a Collection Arrangement” performance indicator. We found that SSA did notuse a standard change control process for the Common Business Oriented Language(COBOL) Programs used in this indicator’s calculation. The COBOL programs,SSIDBT29 and GOALTOT6, used to calculate the performance measure did not adhereto the SSA system development life cycle process, QA2/SRCOL. In addition, theseprograms are not stored in Endeavor but are maintained in personal libraries, wherethey are not subject to SSA's standard system development life cycle process. By notfollowing the SSA standard change control process, changes to the COBOL programsmay not be authorized, tested, or documented appropriately.

Assessment of Meaningful Performance Measurement

We found that SSA could report more meaningful, outcome-oriented results, bydisclosing complementary measures such as the percentage of debt written off eachyear, percentage of new debt being collected each year, and aging reports. Althoughseveral of these indicators are included throughout the PAR, as presented, theperformance indicators’ did not directly disclose sufficient information to enable anindependent reviewer to assess the overall status of debt collection efforts andprogress.

We further found that SSA could develop a more meaningful indicator by onlymeasuring debt that is eligible to be collected. The current indicators’ denominator




includes newly established debt and debt under appeal, the first and second principaldebt categories, which are not eligible to be collected.

In our Management Letter resulting from the 2004 Financial Statement Audit, we foundSSA did not completely disclose its efforts to improve debt collection in compliance with

the Debt Collection Improvement Act (DCIA) of 1996.

16

SSA disclosed several effortsto improve debt collection, however, the DCIA authorized SSA to implement severalspecific procedures to improve overall collection rates and granted Agencies thediscretion to implement the Act in accordance with policies outlined by theCommissioner.17 SSA has determined that certain tools will be followed and phased inover the next 12-24 months. Clearly linking the indicator to other areas in the PAR anddisclosing SSA’s progress and efforts to improve debt collections using available toolswould make the presentation of the indicator results more meaningful.

We found that SSA did not develop annual, incremental improvement goals for theseperformance indicators that supported a clear linkage to the strategic objective to

“Prevent fraudulent and erroneous payments and improve debt management.”

18

SSAmanagement stated that the Agency sets “maintenance goals” and seeks to achieve thesame level of performance as the previous year. Although this indicator was notcreated as part of SSA's Program Assessment Rating Tool (PART) efforts, OMB PARTguidance states, “Goals should be relative to an established baseline, have clear timeframes and targets, and challenge program managers to continuously improve programperformance.”19 SSA management did not provide an explanation in the PAR to justifytheir decision to not establish incremental improvement goals as described in the OMBPART guidance.

16Public Law Number 104-134.

17Although the DCIA authorizes SSA to implement the debt collection tools cited in the statute, it does not

require SSA to do so. In addition, this Management Letter comment was closed as a result of testingcompleted during the FY 2005 Financial Statement Audit.

18Social Security Administration Performance and Accountability Report Fiscal Year 2004, p. 85.

19 http://www.whitehouse.gov/omb/budintegration/part_assessing2004.html, Instructions for the Program Assessment Rating Tool (Section - II. Strategic Planning, Elements of Yes Answer).




CONCLUSION AND RECOMMENDATIONS

SSA should have adequate controls over the generation, calculation, and reporting ofperformance data to ensure a reliable measurement of performance.20 Equally

important is the development of performance indicators that meaningfully provide anassessment of the outcome of efforts to improve collection of outstanding debt.Accordingly, for each performance indicator, we recommend SSA:

1. Improve the documentation of the performance indicators included in this report.The documentation should be complete and accurate to define all of the stepsperformed, accountability for measurement, and all internal controls in operationused to calculate and report the indicator results in the PAR. SSA managementshould annually update and take ownership of the documentation to enable anindependent assessment that complies with OMB A-123 guidance.

2. Retain the detailed data used to calculate the performance indicators results that arereported in the PAR.

3. Continue to implement the Standardized Security Profile Project to restrict access topersonnel that should not have the ability to directly modify, create or delete thedatasets used to calculate the indicators’ results.

4. Provide additional links to other sections of the PAR that discuss progress made(e.g. implementation of new debt collection tools) indicating increased alignment withlaws and regulations that are related to the performance indicators’ results.

5. Provide additional transparency of overall debt collection efforts, link the existingindicators to complementary debt collection measures available both in the PAR andinternal management reports. This linkage should clearly disclose the effect of suchinformation on the indicators’ calculation and the overall progress of debt collectionefforts. The PAR may also include a graphical representation, e.g. a pie chart, toidentify annual changes in each debt collection category.

6. SSA should consider revising the existing performance measure calculation toinclude in its denominator only debt that is eligible to be collected. Newlyestablished debt and debt under appeal, the first and second principal debtcategories, are not eligible for collection. We suggest that the calculation's

numerator include debt in a collection arrangement, the third principal debt category,and the denominator include both debt in a collection arrangement and debt not in acollection arrangement, the fourth principal debt category. This will create a moremeaningful measure that demonstrates the percent of all debt that was eligible to becollected that resulted in a debt collection arrangement.

20 This includes implementing effective controls, as previously reported in our FY 2004 FinancialStatement Audit for the SSI program, for overpayment waivers to provide additional assurance on thevalidity of the performance indicator calculation.




7. Enhance disclosure of SSA’s decision not to establish incremental improvementgoals that results in a departure from OMB guidance.

Specific to the performance indicator “Percent of Supplemental Security Income (SSI)

Debt in a Collection Arrangement,” we recommend SSA:

8. Ensure COBOL programs adhere to the SSA System Development Life Cycle. Inaddition, the programs should be maintained in a version control system in the eventthe program must be backed out and reinstalled.

AGENCY COMMENTS

The Agency agreed with four of our recommendations (numbers 1, 3, 5 and 8) anddisagreed with four recommendations (numbers 2, 4, 6 and 7). For recommendationnumber 6, SSA considered our suggested change to the performance measure

calculation. However, SSA decided to include all unresolved debt in the formula to beconsistent with other Agency reports related to outstanding debt, such as the quarterlyTreasury Report on Receivables to the Treasury Department. Similarly, forrecommendation number 7, SSA is currently developing new performance measures aspart of the President’s Management Agenda initiative on improper payments.

PWC RESPONSE

We appreciate the Agency’s comments and consideration of our recommendations. Inregard to recommendation number 2, we recognize some of the technical challengesassociated with maintaining such detailed data and we understand the Agency must

weigh the cost and benefit of retaining source data. Although we will incorporate real-time audit approaches in ongoing performance measure reviews, these alternative auditsteps will not totally negate the Agency’s need to retain detailed documentation. Assuch, we continue to believe the Agency needs to develop methods to retain thedetailed data used to calculate the performance indicator results reported in the PAR.The lack of data inhibits an independent verification of the performance metricsreported in the PAR.

For recommendation 4, we acknowledge that SSA’s PAR contains some linkage toother sections of the PAR with information related to outstanding debt. However, webelieve this linkage can be improved. To that end, we have revised recommendation 4

to reflect that we are recommending additional linkage.

With respect to recommendation 6, we consider this closed as SSA considered therecommendation and made a business decision to do otherwise. In the event that thenewly developed performance indicators referenced in the Agency’s response torecommendation 7 depart from OMB guidance for goal development, we recommendthat SSA ensure sufficient disclosure of the business reasons for such departures areincluded in the PAR.




Appendices APPENDIX A – Acronyms

APPENDIX B – Scope and Methodology

APPENDIX C – Process Flowcharts

APPENDIX D – Agency Comments

Performance Indicator Audit: Outstanding Debt (A-02-05-15116)



Appendix A

Acronyms

COBOL Common Business Oriented Language

DCFAM Deputy Commissioner of Finance, Assessment, and Management

DCIA Debt Collection Improvement Act

DI Disability Insurance

FY Fiscal Year

GAO Government Accountability Office

GPRA Government Performance and Results Act

MBR Master Beneficiary Record

MIICR Management Information Initial Claims Record

MSSICS Modernized Supplemental Security Income Claims SystemsNIST National Institute of Standards and Technology

OASI Old Age and Survivors Insurance

OASDI Old Age, Survivors, and Disability Insurance

OMB Office of Management and Budget

OQA Office of Quality Assurance and Performance Assessment

OSM Office of Strategic Management

PAR Performance and Accountability Report

PART Program Assessment Rating Tool

ROAR Recovery of Overpayments & Accounts ReceivableSSA Social Security Administration

SSI Supplemental Security Income

SSIPT Supplemental Security Income Processing Time

SSIRMS Supplemental Security Income Records Maintenance System

SSR Supplemental Security Record

U.S.C. United States Code

Performance Indicator Audit: Outstanding Debt (A-02-05-15116)



Appendix B

Scope and Methodology

We updated our understanding of the Social Security Administration's (SSA)

Government Performance and Results Act (GPRA) processes. This wascompleted through research and inquiry of SSA management. We alsorequested SSA to provide various documents regarding the specific programsbeing measured as well as the specific measurement used to assess theeffectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing, including testing ofsource documentation, we performed the following:

• Reviewed prior SSA, Government Accountability Office, Office of theInspector General and other reports related to SSA’s GPRA performance

and related information systems.• Met with the appropriate SSA personnel to confirm our understanding of

the performance indicator.• Flowcharted the process. (See Appendix C).• Tested key controls related to manual or basic computerized processes

(e.g., spreadsheets, databases, etc.).• Conducted and evaluated tests of the automated and manual controls

within and surrounding each of the critical applications to determinewhether the tested controls were adequate to provide and maintainreliable data to be used when measuring the specific indicator.

• Identified attributes, rules, and assumptions for each defined data element

or source document.• Identified and extracted data elements from relevant systems and

obtained source documents for detailed testing selections and analysis.• Recalculated the metric or algorithm of key performance indicators to

ensure mathematical accuracy.• For those indicators with results that SSA determined using computerized

data, we assessed the completeness and accuracy of that data todetermine the data's reliability as it pertains to the objectives of the audit.

As part of this audit, we documented our understanding, as conveyed to us byAgency personnel, of the alignment of the Agency’s mission, goals, objectives,

processes, and related performance indicators. We analyzed how theseprocesses interacted with related processes within SSA and the existingmeasurement systems. Our understanding of the Agency’s mission, goals,objectives, and processes were used to determine if the performance indicatorsappear to be valid and appropriate given our understanding of SSA’s mission,goals, objectives and processes.

Performance Indicator Audit: Outstanding Debt (A-02-05-15116) B-1



We followed all performance audit standards in accordance with generallyaccepted government auditing standards.

In addition to the steps above, we specifically performed the following to test theindicators included in this report:

PERCENT OF OUTSTANDING OLD AGE, SURVIVORS, AND DISABILITY INSURANCE (OASDI) DEBT IN A COLLECTION ARRANGEMENT

• Performed an application review over the Title II systems (ModernizedClaims System, Retirement, Survivors & Disability Insurance Accountingand Recovery of Overpayment, Accounting and Reporting (ROAR)).

• Performed a general controls review over the environment that supportsthe Title II applications and ROAR.

• Inquired of the interface process from the Master Beneficiary Record

(MBR) to ROAR, including resolution of rejected records.• Inspected a selection of interfaces between the MBR/ROAR and

performed a count of records.• Reviewed the code to create the DEBTCATC program for

reasonableness.

PERCENT OUTSTANDING SUPPLEMENTAL SECURITY INCOME (SSI) DEBT IN A COLLECTION ARRANGEMENT

• Performed an application review over the Title XVI systems (Modernized

Supplemental Security Income Claims Systems and SupplementalSecurity Income Records maintenance System).• Performed a general controls review over the environment that supports

the Title XVI applications.• Reviewed the Common Business Oriented Language (COBOL) programs

1 and 2 for reasonableness of the data extraction and summarization ofthe SSI Debt Goal Report.

• Inspected users with update access to the program and the SSI Debt GoalReport.

• Inquired of the process to perform program maintenance over the creationof the SSI Debt Goal Report.

• Inspected a selection of program maintenances during the fiscal year andreviewed to ensure that maintenance followed the Federal InformationSystem Controls Audit Manual.

Performance Indicator Audit: Outstanding Debt (A-02-05-15116) B-2



Appendix C

Percent of Outstanding OASDI Debt in a Collection

Arrangement - 2004 Process Flowchart

Performance Indicator Audit: Outstanding Debt (A-02-05-15116) C-1




Percent of Outstanding OASDI Debt in a Collection

Arrangement - 2004 Process Flowchart Narrative

• The Title II overpayment is detected by Social Security Administration

(SSA) systems or SSA Operations.

• An overpayment record is established on the MBR and ROAR.

• A notice is sent to the beneficiary and an agreement is established for

repayment of the outstanding debt.

• A program designed by Office of Quality Assurance and Performance

Assessment called DEBTCATC is run against a back-up copy of the

ROAR master file and produces monthly debt collection information whichis used to calculate the performance measure. Using the DEBTCATC

program, the amounts for each principal category of debt (new debt, debt

under appeal, debt in a collection arrangement, and debt not in a

collection arrangement) are summed and the portion in a collection

arrangement is determined by dividing the amount in that category by the

total amount of unresolved debt (i.e. the sum of the four principal debt

categories).

• The DEBTCATC report is used as an update to a 12-month rolling

average done by manual calculation. The 12-month rolling average is

calculated by dropping off the percent of dollars in collection for the

earliest month, adding the percent for the latest month and dividing the

total by 12. The result of that calculation is the goal achievement for the

current month. The process of calculating the goal is repeated each

month.

• DCFAM provides the information to OSM for the PAR.



Percent of Outstanding SSI Debt in a Collection

Arrangement - 2004 Process Flowchart





Percent of Outstanding SSI Debt in a Collection

Arrangement - 2004 Process Flowchart Narrative

•

The Title XVI overpayment is detected by SSA systems or SSA Operations.• An overpayment record is established on the SSR.

• A notice is sent to the beneficiary and an agreement is established for repayment

of the outstanding debt.

• A COBOL program called SSIDEBT29 is run against all 20 national file segments

of the SSR to determine the status of debts.

• A second COBOL program is run to total the 20 national file segments and

produces monthly debt collection information on the SSR National Debt Goal

Report. The amounts for each principal category of debt (new debt, debt under

appeal, debt in a collection arrangement, and debt not in a collection

arrangement) are summed and the portion in a collection arrangement is

determined by dividing the amount in that category by the total amount of

unresolved debt (i.e. the sum of the four principal debt categories).

• The SSI National Debt Goal report is used as an update to a 12-month rolling

average done by manual calculation. The 12-month rolling average is calculated

by dropping off the percent of dollars in collection for the earliest month, adding

the percent for the latest month and dividing the total by 12. The result of that

calculation is the goal achievement for the current month. The process ofcalculating the goal is repeated each month.

• DCFAM supplies the information to the OSM for PAR.



Appendix D

Agency Comments



SOCIAL SECURITY

MEMORANDUM

December 19, 2005 Refer To: S1J-3

To: Patrick P. O'Carroll, Jr.Inspector General

From: Larry W. Dye /s/ Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report, "Performance Indicator Audit: OutstandingDebt" (A-02-05-15116)--INFORMATION

We appreciate OIG’s efforts in conducting this review. Our comments on the draft report’s

recommendations are attached.

Please let me know if you have any questions. Staff inquiries may be directed toCandace Skurnik, Director, Audit Management and Liaison Staff, at extension 54636.

Attachment:

SSA Response

Performance Indicator Audit: Outstanding Debt (A-02-05-15116) D-1




COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL DRAFT REPORT,

"PERFORMANCE INDICATOR AUDIT: OUTSTANDING DEBT" (A-02-05-15116)

Thank you for the opportunity to review and comment on the draft report.

Our responses to the specific recommendations are provided below.

Recommendation 1

Improve the documentation of the performance indicators (PI) included in this report. The

documentation should be complete and accurate to define all of the steps performed,

accountability for measurement, and all internal controls in operation used to calculate and report

the PI results in the Performance and Accountability Report (PAR). SSA management should

annually update and take ownership of the documentation to enable an independent assessment

that complies with the Office of Management and Budget (OMB) A-123 guidance.

Response

We agree. As stated in the report on page 4, footnote 13, we provided updated documentation in

April 2005 to adhere to this recommendation. We will continue to review our policies and

procedures to ensure proper documentation for each fiscal year (FY).

Recommendation 2

Retain the detailed data used to calculate the PIs results that are reported in the PAR.

Response

We disagree. Satisfying this recommendation would require SSA to preserve and maintain,

among other things, data transactions, source code, multiple versions of software and the

operating system in use during the potential audit review period. Staff would then need to be

available to reconstruct all this to support an audit. Further, OMB’s Circular A-11, section

203(f) states “Performance data need not be perfect to be reliable, particularly if the cost and

effort to secure the best performance data will exceed the value of any data so obtained.”




During the course of the audit, we offered to have the PricewaterhouseCoopers (PwC) auditors

observe the programmers as the goals reports were being generated. PwC also had the

opportunity to conduct their own "real-time" calculation of the performance measure (at or about

the time of the month when the performance measures were being produced) using the programs

and specifications that they were provided. The PwC auditors choose not to do so.

Recommendation 3

Continue to implement the Standardized Security Profile Project (SSPP) to restrict access to

personnel that should not have the ability to directly modify, create or delete the datasets used to

calculate the PIs results.

Response

We agree. The Agency recently reached a critical juncture in the limitation of programmer

access to its most high-risk systems, thus lifting a reportable condition assessment. SSA will

continue to implement and monitor the progress of SSPP. Additionally, SSA will target, for

implementation of SSPP, those systems related to the development of Government Performance

and Results Act performance measures.

Recommendation 4

Clearly link discussions in other sections of the PAR that discuss progress made (e.g.,

implementation of new debt collection tools) indicating increased alignment with laws and

regulations that are related to the PIs results.

Response

We disagree. We believe that the information in the PAR already contains the clear linkage that

is being recommended. In the Debt Management summary on page 218 of the FY 2004 PAR,

SSA’s progress in implementing the various debt collection tools authorized by law are clearly

discussed, as is the performance indicator.




Recommendation 5

Provide additional transparency of overall debt collection efforts; link the existing indicators to

complementary debt collection measures available both in the PAR and internal management

reports. This linkage should clearly disclose the effect of such information on the PIs calculation

and the overall progress of debt collection efforts. The PAR may also include a graphical

representation, e.g., a pie chart, to identify annual changes in each debt collection category.

Response

We partially agree. We agree that the Agency can better link the performance indicator to other

relevant portions of the PAR and how it relates to the President's Management Agenda (PMA)

initiative of "Improper Payments". We believe we already have an appropriate level of

information supporting our debt collection performance indicators in the PAR, and do not

believe it is necessary or appropriate to expand this external document to discuss internal

management reports or add additional graphics on other measures.

Recommendation 6

Consider revising the existing PI calculation to include in its denominator only debt that is

eligible to be collected. Newly established debt and debt under appeal, the first and second

principal debt categories, are not eligible for collection. OIG suggests that the calculation’snumerator include debt in a collection arrangement, the third principal debt category, and the

denominator include both debt in a collection arrangement and debt not in a collection

arrangement, the fourth principal debt category. This will create a more meaningful measure that

demonstrates the percent of all debt that was eligible to be collected that resulted in a debt

collection arrangement.

Response

We disagree. We considered this option when the measures were being developed, but it was

decided to include all unresolved debt in the formula to be consistent with what the Agency

reports elsewhere as outstanding debt (i.e., the quarterly Treasury Report on Receivables to the

Treasury Department). Additionally, the measures were developed in consultation with OMB,

which is fully aware and in agreement with them.




Recommendation 7

Enhance disclosure of SSA’s decision not to establish incremental improvement goals that

results in a departure from OMB guidance.

Response

We disagree. SSA is in the process of developing increased performance measures for debt

collection as part of the PMA initiative on "Improper Payments".

Recommendation specific only to performance indicator, “Percent of Outstanding SSI Debt in a

Collection Arrangement”

Recommendation 8

Ensure COBOL programs adhere to the SSA Systems Development Life Cycle. In addition, the

programs should be maintained in a version control system in the event the program must be

backed out and reinstalled.

Response

We agree. We will ensure that appropriate controls are established to make certain that no datais lost in the event the program must be backed out and reinstalled.



Overview of the Office of the Inspector General

The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI),

Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office

of Resource Management (ORM). To ensure compliance with policies and procedures, internalcontrols, and professional standards, we also have a comprehensive Professional Responsibility

and Quality Assurance program.

Office of Audit

OA conducts and/or supervises financial and performance audits of the Social Security

Administration’s (SSA) programs and operations and makes recommendations to ensure

program objectives are achieved effectively and efficiently. Financial audits assess whether

SSA’s financial statements fairly present SSA’s financial position, results of operations, and cash

flow. Performance audits review the economy, efficiency, and effectiveness of SSA’s programsand operations. OA also conducts short-term management and program evaluations and projects

on issues of concern to SSA, Congress, and the general public.

Office of Investigations

OI conducts and coordinates investigative activity related to fraud, waste, abuse, and

mismanagement in SSA programs and operations. This includes wrongdoing by applicants,

beneficiaries, contractors, third parties, or SSA employees performing their official duties. This

office serves as OIG liaison to the Department of Justice on all matters relating to theinvestigations of SSA programs and personnel. OI also conducts joint investigations with other

Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General

OCCIG provides independent legal advice and counsel to the IG on various matters, including

statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on

investigative procedures and techniques, as well as on legal implications and conclusions to be

drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary

Penalty program.

Office of Resource Management

ORM supports OIG by providing information resource management and systems security. ORM

also coordinates OIG’s budget, procurement, telecommunications, facilities, and human

resources. In addition, ORM is the focal point for OIG’s strategic planning function and the

development and implementation of performance measures required by the Government

Social Security: A-02-05-15116

Documents

Transcript of Social Security: A-02-05-15116