Audit Guideline

Document 11 Revision 17

Page 1 of 31

JOINT AUDIT AND CHECKLIST IMPLEMENTATION GUIDELINE

This guideline has been developed to assist the audit team in the utilization of the NUPIC Audit Checklist. The current edition of this checklist utilizes the philosophy and principles of EPRI/NP-6630 - "Guideline for Performance-Based Supplier Audits (NCIG-16)." Each audit team member is required to thoroughly understand and adopt the NUPIC philosophy for conducting PBSA(s) of suppliers.

A performance-based audit examines both the technical and quality requirements of a vendor's manufacturing/service activities to assess the adequacy and effectiveness of the vendor's quality program. This review begins with an analysis of the vendor's contract review process, and an examination of design, commercial grade dedication, software QA, procurement and material controls associated with specific utility order(s). The team assessment progresses to field (shop) observations of fabrication/assembly/special processes/tests/inspection activities for the same (or similar) utility procurements. Investigations of associated calibration and handling/storage/shipping activities and review of associated documentation completes the performance-based audit. Lastly, the unreviewed sections of the vendor's quality program(s) are assessed based on data gathered during the aforementioned performance-based review. The PBSA Worksheet is used during the planning, performance, and reporting phases of all NUPIC audits and is to be included in the final audit report package when it is issued. The Audit Checklist delineates what activities are to be examined within each section and how the referenced data sheets are to be utilized to record the objective evidence reviewed for each section. The corresponding guidance delineates how this activity should be accomplished. It should be noted that figures do not exist for every checklist question. Items not referenced on figures are to be fully addressed with appropriate examples of objective evidence documented in the Assessment/Summary sections of the checklist. The following notes apply to the NUPIC checklist:

° Checklist Item 6.5 does not apply to software because shipping of software is addressed in Item 4.7.

° The column entitled "audit scope" on page 1 of the Summary Sheet is to contain

check marks for those codes and standards for which compliance was evaluated during the audit, based on utility input and the supplier’s QA Manual.

° of the Summary Sheet illustrates recommended sections to be used when

auditing engineering or field services suppliers. E=Engineering, F=Field Services.

° Audit Checklist sections (including figures) deemed as not applicable can be discarded (i.e., do not have to be included in the final audit package), provided the section's non-applicability is clearly documented under Comments on Page 2 of the Summary Sheet . The following are two examples illustrating the correct use and incorrect use of this requirement:


Page 2 of 31


CORRECT

The supplier’s scope of work may not include Design, Software Quality Assurance

or Field Services. On the appropriate line for these sections on page 2 of the Summary Sheet enter this comment: “N/A – The supplier’s scope of work does not include this activity.” Therefore, this section has been eliminated from the checklist.

INCORRECT “N/A – The supplier has not performed any of these activities since the previous

audit.” If this is determined to be the case, the appropriate controls for those applicable activities must still be addressed to show that an adequate program is in place should the occasion arise when these activities require implementation.

° Checklist items deemed, as "not applicable" require documented justification

defined in the audit checklist's "Assessment/Summary" section. Associated data sheet(s) need only be identified as "N/A." Below are two examples illustrating the correct use and incorrect use of this requirement:

CORRECT

Checklist Item 5.5 relates to acceptance of material from an ASME Material Organization. If the supplier’s scope of work is electrical components, the justification entered in the Assessment/Summary section for this item should be: “N/A – The supplier’s scope of work is electrical and does not include ASME material.” Therefore, this checklist attribute is not applicable.

INCORRECT “N/A – The supplier only procures ASME material from ASME Material

Organizations/Certificate holders.” This justification does not support the use of N/A since the supplier procures ASME material from ASME Certificate holders. An evaluation is therefore required to assess the supplier’s method of acceptance of material from ASME Certificate holders.

o Justification for the use of N/A in the columns of the checklist figures/data sheets is

not required when it is apparent the requested information is not applicable.

EXAMPLE: Figure 5B/12 – If five findings were issued during an external or internal audit and all were still open, the corrective action verification and method date column would be N/A.

o Any additional questions deemed necessary for the audit can be added to the

checklist prior to or during the audit by the Audit Team Leader.

o The use of the symbol "/" in this guideline or on the Audit Checklist signifies an "and/or" statement.

° The sequence in which the Audit Checklist is executed may be varied and is to be

determined by the Audit Team Leader.


Page 3 of 31


o The first checklist item within each section requires procedures, instructions, and

drawings, including their revision/date to be identified within the assessment/summary section of the checklist. Verification of revision control on a sample of these documents shall be performed and documented by the auditor assigned to each checklist section. Revision 12 of the checklist removed Figure 9 and placed this requirement in the individual checklist sections. Section 9 is now intended to address procedural control of documents.

° All figures must be annotated; blanks are not acceptable.

° Figures requiring a “Yes or No” to be entered should include an explanation when

“No” is entered. ° Each checklist item must be evaluated to determine if Field Services should be

addressed. If applicable, the assessment should clearly address the adequacy of controls for this area as it applies to Field Services.

If the supplier controls Field Services under the same quality program which is implemented for the control of in-house activities, examples of the adequacy and implementation of the controls shall be documented in each applicable section of the checklist.

In those instances when the supplier has a separate quality program for Field Services, the controls prescribed by the separate quality program should be evaluated and addressed in the applicable sections of the checklist in addition to other quality program requirements.

The following areas may be considered when evaluating Field Services:

Contract Review Software Quality Assurance Procurement Fabrication/Assembly Activities Material Control Special Processes Test and Inspections/Calibration Document Control/Adequacy Organization/Program Nonconformance/Corrective Action Personnel Qualifications/Training Records

Checklist Item 15.1 is intended to provide a description of the Field Services provided by the supplier and an overall assessment verifying that the applicable sections of the checklist adequately evaluate Field Services. The assessment should also list the checklist sections determined to be applicable to the Field Services provided by the supplier.


Page 4 of 31


° When completing the Assessment/Summary section of a Checklist item, the

following elements should be considered:

° Address both the establishment and implementation of the program controls. State or reference where these controls are established.

° Include procedure verified and it’s revision/date ° Describe who/how/where/when/these controls are implemented. ° Describe how you verified implementation (i.e. through observation of in-

process work activities, documentation review, interview, etc). Include a brief description of what you looked at and the sample size chosen (an explanation of a small sample size may be warranted or needs to be qualified) or make reference to the Figure where the objective evidence is recorded.

° Finally, provide your assessment. (Ex. Controls are/are not implemented and are/are not adequate.) The following is an example of an acceptable assessment/summary for a checklist question:


Page 5 of 31


Example of Acceptable Assessment/Summary

SECTION 5 - PROCUREMENT

METHOD OF VERIFICATION 5.3 Verify that measures are established and implemented for the evaluation, selection and assessment of suppliers (including

distributors and calibration, NDE, testing labs, software suppliers, heat treatment services suppliers) consistent with the importance, complexity and quality of the product or service. Verify the supplier’s program addresses audits of Appendix B suppliers (and surveys for commercial grade suppliers, if applicable).

a) Verify evaluations are performed prior to award of contract, and at the specified frequency.

b) Verify that the scope of approval of the sub-supplier is commensurate with the requirements of the procurement documents.

c) Verify that only approved suppliers are used.

(Document O.E. on Figure 5A)

NOTE: Content of CGI surveys will be addressed in Section 3.

Appendix B/ANSI N45.2 Ref: (7/8)

ASME Section III

NQA-1 Supplement 7S-1

Vendor Quality Manual Ref.: QA Manual Section 7

RESULTS: Satisfactory

ASSESSMENT/SUMMARY: Procedure 789, Revision 4, establishes the requirements for supplier qualification and includes the requirements of a), b) and c) above. Suppliers are evaluated and qualified by audit or survey as applicable. The QA Manager conducts supplier audits/surveys at the sub-suppliers facility. An Approved Suppliers List (ASL) is maintained which indicates supplier name, location, item/service (scope), any restrictions, and audit due date. The ASL contains ASME, safety related (non-code) and commercial suppliers qualified by audit/survey. Suppliers listed on the ASL include material suppliers, calibration labs, NDE service suppliers and testing labs. Distributors, software suppliers, and heat treatment service suppliers are not used.

a) Suppliers listed on the ASL are approved via audit/survey prior to award of contract and re-audited every 3 years. The audit records reviewed included a sample of each of the categories discussed above. Note – the audit/survey reports listed on Figure 5B were reviewed and comprise approximately 15% of those performed since the last NUPIC audit.

b) Figure 5A lists the PO’s that were compared to the ASL and the audit/survey reports listed on Figure 5B. The PO’s were issued to suppliers within the approved scope of supply.

c) Conversation with the QA Manager and review of PO’s listed on Figure 5A confirmed that safety related suppliers (material and services) are only qualified via audit. Commercial material may be purchased from unapproved suppliers, however additional verifications are performed via the dedication process if the material is used in safety related applications.

Revision 4 was verified to be the current revision of the procedure.

The supplier’s written process for control of sub-suppliers is acceptable and adequately implemented.

RESULTS: Satisfactory


Page 6 of 31


SECTION 3 – COMMERCIAL GRADE DEDICATION – ADDITIONAL GUIDANCE Checklist Item 3.2 - 3.4 Items 3.2 - 3.4 are intended to evaluate whether the supplier has established and implemented program controls for the dedication of commercial grade items/services. The dedication process should include an evaluation of the item/service, the identification of critical characteristics and the method for verifying the critical characteristics, as appropriate. Special consideration should be given when the scope of supply includes items that are subject to seismic and/or environmental qualification requirements. When seismic qualification requirements apply, the supplier’s commercial grade dedication package should include seismic critical characteristics (those characteristics of the item that support the ability of the item itself or the host equipment to perform safety functions during and after a design basis accident). In some cases, this may be only that the configuration is the same and weight of the item has not increased and material is the same. Seismically sensitive items (an item or class of items that has the potential to affect the seismic adequacy of the item itself or host equipment) such as relays, switches and contactors may have additional seismic critical characteristics that may need to be considered during the dedication process. Examples of seismic critical characteristics for seismically sensitive items include:

• Mass and strength of the materials • Configuration and dimensions/tolerances of the item and certain subcomponents • Spring constant • Contact bounce time • Operate time

Additional guidance concerning the dedication of seismically sensitive items can be found in EPRI Technical Report TR-112579 “Critical Characteristics for Acceptance of Seismically Sensitive Items (CCASSI). When environmental qualification requirements apply, the supplier’s commercial grade dedication package should include environmental critical characteristics (those characteristics of the item that support the ability of the item itself or the host equipment to perform safety functions during and after a design basis accident). Material characteristics (such as chemical composition and material strength) are often verified to ensure that qualification is maintained. The original seismic and environmental qualification of the parent equipment must be maintained as items within those components are replaced. Several methods are typically employed to verify that qualification is maintained, including the following: • Conducting the original testing on a sample of replacement items to the original qualification tests (for example, loss of coolant accident chamber, shake-table, etc.) • Inspecting (sometimes requiring disassembly of) items to ensure that the item’s design has not changed from that of the item originally tested. • Conducting tests that verify the item’s seismic critical characteristics (that is, functional tests, measuring spring force, etc.) It is important to remember that original qualification requirements apply at the part level as well as at the component level. If the original qualification report for a transmitter qualified ethylene propylene diene monomer (EPDM) as the suitable O-ring material, a critical characteristic of O-ring material should be identified in dedication packages for both replacement transmitters and replacement O-rings, and the dedication should verify that the O-ring material is EPDM. Another example would be


Page 7 of 31


electromechanical components (e.g., relays, switches, contactors, etc.) that are classified as seismic sensitive items and could be supplied as individual parts or as part of an assembly to be installed in the host equipment such as on a printed circuit board. In this instance, the dedication of the assembly must address the seismic critical characteristics for any seismically sensitive devices contained in the assembly that have a safety function or could adversely affect the safety function of the host equipment during a design basis accident. Audit team members should pay particular attention to seismic and environmental qualification requirements when auditing suppliers (especially third-party qualifiers) who might not have been involved in the original qualification effort or be fully aware of the original equipment qualification requirements. The technical specialist member of the audit team may be a source of knowledge when evaluating dedication activities. Examples: The aforementioned O-ring is a part of an environmentally qualified transmitter. As with most O-rings, part number, configuration and dimensions are critical. In dedicating this O-ring for a qualified transmitter, material may be a key critical characteristic to ensure the transmitter can withstand the radiation, temperature and/or humidity for which it was environmentally qualified. If the O-ring was EPDM in the qualified transmitter, the dedication should verify that the O-ring material is EPDM. If equipment was environmentally qualified using insulated terminal lugs or a phenolic part, material may be key to environmental qualification. Unless the engineering evaluation includes justification for shielding from the environment or other justification that material is not affected, the dedication should verify that the replacement terminal lug insulation material or the phenolic part is the same as the original. A small valve with an operator was originally seismically qualified. Although the replacement operator is the same part number, has the same configuration and functionally performs the same, the housing, mounting bracket and fasteners are now much heavier to make the operator more seismically rigid. However this heavier housing and hardware changes the weight (and perhaps the moment) of the valve assembly which may affect the seismic qualification. For the first two examples, chemical testing (Method 1) may be needed by the supplier to verify materials or perhaps the supplier may need to perform a commercial grade survey (Method 2) of the subsupplier to verify material verification. For the third example the valve assembly may need to be seismically re-analyzed to determine if qualification has been affected. Perhaps the assembly will need a new seismic qualification (Method 1).


Page 8 of 31


CHECKLIST SECTION 4 TERMS/DEFINITIONS Activity – Software Life Cycle Phase Acceptance Testing - Validation. The process of exercising or evaluating a system or system

component by manual or automated means to assure that it satisfies the specified requirements and to identify differences between expected and actual results in the operating environment.

Acceptance Test Plan (ATP) - Describes the plan for acceptance testing of the software

application. This would usually be published as a separate document, but might be published with the system test plan as a single document. It should include how the application properly handles abnormal conditions and events as well as credible failures; does not perform adverse unintended functions; and does not degrade the system either by itself, or in combination with other functions or configurations.

Acceptance Test Report (ATR) - Describes the activities and results for acceptance testing of

the software application based on the Acceptance Test Plan (ATP). Baseline - A specification or product that has been formally reviewed and agreed upon, that

thereafter serves as the basis for use and further development, and that can be changed only by using an approved change control process.

Configuration Control - See Configuration Management Configuration Item - A collection of hardware or software elements treated as a unit for the

purpose of configuration control. Configuration Management (software) - The process of identifying and defining the configuration

items in a system (i.e., software and hardware), controlling the release and change of these items throughout the system’s life cycle, recording and reporting the status of configuration items and change requests.

Computer Program - A combination of computer instructions and data definitions that enables

computer hardware to perform computational or control functions. Development Cycle - The activities that begin with the decision to develop a software product and

end when the software is delivered. The software development cycle typically includes the following activities: design requirements, design, implementation, testing and sometimes installation.

Disaster Recovery - The ability to regain the software and/or operating environment consistent

with its required quality level in a timely manner. Error - A condition deviating from an established baseline, including deviations from the current

approved computer program and its baseline requirements. Firmware - Software (programs or data) that has been written onto read-only memory (ROM).

Firmware is a combination of software and hardware. ROMs, PROMs and EPROMs that have data or programs recorded on them are firmware.


Page 9 of 31


Platform - The underlying hardware or software for a system. The platform defines a standard

around which a system can be developed. Once the platform has been defined, software developers can produce appropriate software and managers can purchase appropriate hardware and applications. The term is often used as a synonym of operating system. (e.g. Windows 95, Windows NT, Unix, VAX, DOS etc.)

Product Baseline - In configuration management, the initial approved technical documentation,

(including, for software, the source code listing) defining a configuration item during the production, operation, maintenance, and logistic support of its life cycle.

PROM - An acronym for programmable read-only memory. A PROM is a memory chip on which

data can be written only once. Once a program has been written onto a PROM, it remains there forever. Unlike RAM, PROMs retain their contents when the computer is turned off. The difference between a PROM and a ROM (read-only memory) is that a PROM is manufactured as blank memory, whereas a ROM is programmed during the manufacturing process. To write data onto a PROM chip, you need a special device called a PROM programmer or PROM burner. The process of programming a PROM is sometimes called burning the PROM. An EPROM (erasable programmable read-only memory) is a special type of PROM that can be erased by exposing it to ultraviolet light. Once it is erased, it can be reprogrammed. An EPROM is similar to a PROM, but requires only electricity to be erased.

Requirement - (1) A condition or capability needed by a user to solve a problem or achieve an

objective. (2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documents. (3) A documented representation of a condition or capability as in (1) or (2).

Review (Verification) – A process or meeting during which a work product, or set of work products,

is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. Types include code review, design review, formal qualification review, requirements review, test readiness review.

Software - Computer programs and associated documentation and data pertaining to the operation

of a computer system. Software Design Document (SDD) - The record of the final system design. This document

provides the final stages of the implementation of the requirements described in the Software Requirements Specification.

Software Design Requirements (SDR) - Address technical and software engineering

requirements and shall be traceable throughout the software life cycle. Software Life Cycle - The activities that comprise the evolution of software from conception to

retirement. The software life cycle typically includes the software development cycle and the activities associated with operation, maintenance, and retirement.

Software Quality Assurance Plan (SQAP) - A plan for the development of software products

necessary to provide adequate confidence that the software conforms to established requirements.


Page 10 of 31


Software Requirements Specification (SRS) - Specifies the requirements for a Computer

Software Configuration Item (CSCI) and the methods to be used to ensure that each requirement has been met.

Validation - Acceptance Testing Verification – Review


Page 11 of 31


SECTION 4 – SOFTWARE – ADDITIONAL GUIDANCE

Checklist Item 4.2 (a) Item 4.2 (a) is intended to evaluate whether the supplier has established program controls for the use of safety-related of software (i.e. software, firmware, digital, etc.) It inquires if the structure for the software lifecycle is suitable for use in the acquisition, development, maintenance, supply, and operation of software. The nature of the software application and how it supports the product(s) of the manufacturer should be explained in this question. If the audit consists of a software developer this should include the policies, procedures, plans, specifications, or work practices that provide the framework for software engineering activities. The complexity and whether the supplier is a user or developer of the software package will dictate how much of the software life cycle is involved. Examples: A software development firm that is providing nuclear safety-related software applications should have an established program that meets the necessary requirements of 10CFR50, Appendix B and a systematic life cycle that addresses design, design verification, implementation, acceptance testing, operation, maintenance, and retirement. A manufacturer of other products and services that utilize software (i.e. pump manufacturer for flow calculations) should have a program in place to address how software is used. This may not include design of software but may include acceptance testing, configuration management, operation, and retirement. The scope of Item 4.2 (a) is intended to be broad. The question is used to determine if the supplier has a program and what activities of the software lifecycle process are involved. This is the responsibility of the auditor to determine which, if any, software applications apply, including digital control components Checklist Item 4.2 (b) This item applies to manufacturers that develop and procure software. Even if a manufacturer only procures software, this question should be answered by discussing how the supplier uses the software when/if the supplier does not have the source code. The documentation may be different but the concept is the same; the application needs to be reviewed at the different lifecycle activities. Again, this question relies on the knowledge and skill level of the auditor. Examples: Requirements Review – The purpose of the requirements review is to gather the customers, users, systems analysts, and management together to establish specification buy-in. This is one of the most cost-effective ways to remove errors within a software application. Design Review – Design reviews are the process of using problem and software knowledgeable inspectors to examine a proposed design to see if it is necessary and sufficient to solve the problem provided. Reviews come in many different levels of formality, from group reviews to individual inspections. Preliminary design reviews check to see of the design is moving in the right direction. Critical design reviews check to see if the design got to the right place.


Page 12 of 31


Peer Code Review - This is a process by which a team of software inspectors performs an in-depth review of a software product or part of a product. It is also known by other terms like code inspection, code verification, or code walkthrough. Checklist Item 4.3 Acceptance testing activity should demonstrate that the application adequately and correctly performs all intended functions. The question applies to acceptance testing whether the supplier developed the code or if the application was purchased. If the supplier performs testing for product baseline then the auditor should perform the evaluation, if not, then only the acceptance testing should be evaluated. Test plans, test cases, and test results should be documented, reviewed, and approved. Examples: Acceptance testing is the culmination of a testing process. Below is a listing of various tests, not all, which an auditor may encounter. Unit Testing – Unit tests are test of individual software components usually conducted with test drivers, which are special code written just to cause the component to execute. Integration Testing – Integration testing is the processes of putting the software pieces together and seeing how well they work together. System Testing – System testing is the first point in the testing process in which the total software product can be reviewed in a realistic setting. Independent Testing – Independent testing is the process of using skilled testers who are not part of the application development team to test the application. Checklist Item 4.4 The intent of this item is to determine how the supplier controls changes to the software product. This should include enhancement requests from the customer, revision to software based on the design requirements, changes to the operating environment, or reported software problems that must be corrected. This may include items such as Change Trackers, which are clerical/management tools for reporting the content and status of software changes; Comparators, which are tools for comparing the differences between one version to another; Documentation Managers, which assist in the production of updated software documentation. Checklist Item 4.5 This item was established to assure that the supplier has developed a program for purchasing software. This includes software development tools for application developers, and software products for manufacturing and service suppliers. Part A of this question is directed at the procurement of safety-related software while Part B of this question is directed at Commercial Grade. Original software design requirements are not always available for procured software such as Commercial-off-the-shelf (COTS). Acceptance testing should be performed on these applications as well. This question is not only stating that the software is tested and accepted, but commercial grade dedication


Page 13 of 31


activities are performed and documented. This should include any critical characteristics defined by the supplier and one of the four (4) methods of dedication described in EPRI NP-5652. Checklist Item 4.6 Item 4.6 verifies that the supplier has a program in place to document errors, which should include Part 21 notification for safety related applications as applicable. Errors are conditions deviating from an established baseline, including deviations from current approved applications and its baseline requirements. Documentation of anomalies or errors found as part of development, after baseline, should be document on Figure 4. Checklist Item 4.7 This item inquires if the supplier has a program for the handling, storage, packaging, preservation, and delivery of the software media. Examples: Repositories should be backed up to protect from unintentional damage. Duplication process should be reviewed to assure that the approved application is transferred to the appropriate media. In some cases a new or upgraded application can be downloaded from a supplier’s File Transfer Protocol (FTP) site or Internet web page.


Page 14 of 31


Section 8 – TESTS, INSPECTIONS, AND CALIBRATION – ADDITIONAL GUIDANCE Checklist Item 8.4 Item 8.4 is intended to evaluate a supplier’s process to identify and disposition fraudulent/counterfeit materials, parts, components, equipment and services from being accepted for use in a safety-related (or quality-related) application. Such fraudulent/counterfeit materials, parts, components, equipment and services could be a factor in the failure of equipment furnished to a nuclear plant in performing its intended safety function. The supplier’s process should include measures to identify and disposition suspect materials, parts, components, equipment and services. Item 8.4 is intended to evaluate the supplier’s process involving detection/identification including inspection (receiving, in-process and final), document review, and testing (receiving, in-process and final). The supplier’s process for identifying and dispositioning suspected fraudulent/counterfeit items should be documented. Document(s) could be procedures and/or instructions which are specific to the process, or it could also be a process description added to existing procedures or instructions. Personnel performing activities that could detect, identify and/or disposition suspected fraudulent/counterfeit items should be given awareness/detection training. In addition to those listed in Item 8.4, examples of process attributes include: • verification of shipping location; • review of shipping documentation; • thorough review of certification for alterations, different font, hand-written changes,

conformance with PO requirements; • inspection of item for evidence of wear, mis-matched parts, ill-fitting parts, suspect item

manufacturer markings/model numbers/serial numbers, multiple lots/batches/heat codes; • comparison to known legitimate item; • testing to verify expected performance characteristics; • testing for material verification; A finding should be written if:

(a) a process has not been undertaken, (b) the process is not documented, (c) affected personnel are not trained or are not aware of suspect/fraudulent/counterfeit

process, or (d) the process is not implemented.

Example of a finding requirement and noncompliance: REQUIREMENT: 10CFR50 Appendix B Criterion V “Instruction, Procedures and Drawings” states that activities affecting quality shall be prescribed by documented instructions, procedures, or drawings, of a type appropriate to the circumstances and shall be accomplished in accordance with these instructions, procedures, or drawings. Instructions, procedures, or drawings shall include appropriate quantitative or qualitative acceptance criteria for determining that important activities have been satisfactorily accomplished.


Page 15 of 31


10CFR50 Appendix B Criterion VII “Control of Purchased Material, Equipment and Services” states that measures shall be established to assure that purchases material, equipment, and services, whether purchased directly or through contractors and sub-contractors, conform to the procurement documents. These measures shall include provisions, as appropriate, for source evaluation and selection, objective evidence of quality furnished by the contractor or subcontractor, inspection at the contractor or subcontractor source, and examination of products upon delivery. NONCOMPLIANCE: Procedural requirements/guidance has not been adequately established for the detection of potentially counterfeit/fraudulent products during performance of product receipt inspection and/or commercial grade item dedication. NRC Information Notice IN 2008-04, IN 89-70, IN 89-70 Supplement 1 and Generic Letter 89-02 should be reviewed for additional information associated with the detection of misrepresented vendor products.


Page 16 of 31


SECTION 1 – CONTRACT REVIEW

Figure 1

COLUMN ITEM GUIDANCE

UTILITY P.O./CONTRACT NO. AND DATE

1.2 Record utility purchase order/contract number and date.

PART NUMBER AND ITEM/SERVICE DESCRIPTION

1.2 Record item part number, item/service description from purchase order reviewed.

P.O./CONTRACT REQUIREMENTS CORRECTLY TRANSLATED TO SUPPLIER DOCUMENTS (Yes/No) (LIST WORK ORDERS, TRAVELERS, DRAWINGS, ETC.)

1.2 Indicate by Yes/No if the requirements were correctly translated. Record supplier document to which technical/quality requirements have been transferred from P.Os. listed in previous columns. (i.e., travelers, drawings, work orders, etc.)

UTILITY APPROVAL OF EXCEPTIONS (Yes/No)

1.2 Indicate by Yes/No if exceptions were approved by the utility. Explain any “No’s”


Page 17 of 31


SECTION 2 – DESIGN

Figure 2


UTILITY/SUPPLIER DESIGN INPUT AND BASES

2.2 Record the utility/supplier bases identified that apply (i.e., codes/code requirements, calculation, EQ qualification report, seismic report, etc.).

SUPPLIER DESIGN DOCUMENT 2.2 Record the document (e.g., drawings, specifications, calculations, etc.) used to translate design requirements.

DESIGN INPUTS CORRECTLY INCORPORATED (Yes/No)

2.2 Indicate by Yes/No if inputs are incorporated. Explain any “No’s”

METHOD OF DESIGN VERIFICATION 2.5 Record the method of design verification (i.e., design review, alternate calculation, or test).

DESIGN CHANGE CONTROL AND REV./DATE

2.6 Record change document used (i.e., ECN's, customer approvals, drawings, etc.) and revision/date.


Page 18 of 31


SAMPLE OF COMPLETED FIGURE 2

SECTION 2 – DESIGN

(FIGURE 2)

UTILITY/SUPPLIER DESIGN INPUT AND BASES

SUPPLIER DESIGN DOCUMENT DESIGN INPUTS CORRECTLY

INCORPORATED

(Yes/No)

METHOD OF DESIGN VERIFICATION

DESIGN CHANGE CONTROL AND

REV./DATE

*2.2 *2.2 *2.2 *2.5 *2.6

ASME Section III, 1980 Edition including Winter Addenda, Class 2 per ANSI B16.34 1977 Special Class; FPL Design Specification PV -156, Rev. 0

(Reference FPL P. O. xxxxxx)

Drawing D-123789, Rev. 2

“Pneumatic Valve, Plug Type, Globe Pattern, 5/16” Dia. Orifice, ANSI 1500 SPL, Bellows Seal, Air to Open Operator”

Yes

Design Review and Test

ECN No. 01 dated 01-04-

03

ASME Section III, IEEE 344-1971, FPL Design Specification PV -156, Rev. 0


Engineering Report

ER-156, Rev. 0

“Pressure Boundary and Seismic Analysis of Air Operated Nuclear Bellows Valve”

Yes Design Review N/A

IEEE 344-1971, FPL Design Specification PV-156, Rev. 0


Calculation

No. SA-564, Rev. 0

“Stress Analysis of Air Operated Nuclear Bellows Valve”

Yes

Design Review N/A

*Refers to applicable question.

* Explain any “No’s”


Page 19 of 31


SECTION 3 – COMMERCIAL GRADE DEDICATION

Figure 3A


ITEM DESCRIPTION P/N, S/N, MODEL NO., SOFTWARE NAME/ID NO., ETC. AS APPLICABLE

3.2, 3.3 and 4.5

Enter description and, as applicable, Software Name/ID number, etc., part number, serial number, model number, etc. of the item evaluated.

CRITICAL CHARACTERISTICS AND METHOD OF DEDICATION

3.2, 3.3 and 4.5

List the critical characteristics identified by the supplier. Report method used for dedication of critical characteristics (i.e., test and inspection, survey, source verification, etc.)

**INSPECTION /TEST PROCEDURE REV./DATE

3,2, 3.3 and 4.5

List procedure number, revision and or date used to perform activities.

**INSPECTOR/TESTER NAME/STAMP 3,2, 3.3 and 4.5

List inspection/test personnel that performed the activity.

**ID NUMBER OF M&TE USED 3,2, 3.3 and 4.5

Record the ID number of the M&TE used.

**RESULTS SAT. OR UNSAT., RECORD NCR NO. IF APPLICABLE

3,2, 3.3 and 4.5

Enter Sat. or Unsat. for results. If Unsat., record NCR number if applicable.

** These attributes are to be addressed only for Method 1. When Method 2 or 3 is employed, Checklist Attribute 3.4 and Figure 3B are to be completed. .


Page 20 of 31


SECTION 3 – COMMERCIAL GRADE DEDICATION

FIGURE 3B (SURVEYS/SOURCE VERIFICATION)**


CGI SUPPLIER NAME, LOCATION AND DATE(S) PERFORMED

3.4, 4.5 List the name and city and state of the supplier where survey or source verification was performed. Also list the date(s) activities were performed.

EVALUATION METHOD (COMMERCIAL GRADE SURVEY OR SOURCE VERIFICATION)

3.4, 4.5 List which evaluation method (CGI survey or source verification) was performed to verify critical characteristics.

SCOPE OF SUPPLY 3.4, 4.5 List the scope of supply for which the supplier was being evaluated (i.e. what component, part, material or service being supplied).

AUDITORS 3.4, 4.5 List the auditor(s) that performed the survey. If source verification was used, list the title or function of personnel performing activity (e.g. inspector, engineer, etc.)

CRITICAL CHARACTERISTICS (CC) VERIFIED

3.4, 4.5 List the specified critical characteristics which were verified by the activity. If there were any CCs .

DO CCs VERIFIED BY THE EVALUATION MATCH THOSE SPECIFIED (Yes/No)

3.4, 4.5 Does the CCs verified by the survey/source verification match those specified to be verified (by the survey/source surveillance)? Enter yes or No. If no, explain in the Assessment/Summary of Attribute 3.4.

** This figure to be completed for Methods 2 and 3 only.


Page 21 of 31


SECTION 4 - SOFTWARE

Figure 4


SOFTWARE PROGRAM – NAME, NO., REV./DATE

4.2, 4.4 List software program by name, number, revision level/date.

METHOD OF ACCEPTANCE TESTING AND DATE

4.2, 4.3 Identify the method used for acceptance testing of the software (i.e., hand calculation, other programs, etc.) including date.

ERROR NOTICE/DATE AND STATUS (OPEN/CLOSED)

4.6 Record Error Notice ID number and date. Indicate Error Notice Status (i.e., Open/Closed)


Page 22 of 31


SECTION 5 - PROCUREMENT Figure 5A


ITEM DESCRIPTION (P/N, S/N, MODEL NO., SOFTWARE NAME)

4.5, 5.2, 5.3 Record procured item including P/N.

SUPPLIER AND LOCATION 4.5, 5.2, 5.3 List the supplier name, division (if applicable) and location (city/state).

P.O. NUMBER AND DATE 4.5, 5.2, 5.3 Record the supplier’s purchase, order and date.

METHOD AND DATE OF SUPPLIER EVALUATION

4.5, 5.3, 5.4 List the method of evaluation including the date (i.e., audit, survey, or source verification, etc.).

SCOPE OF SUPPLIER APPROVAL 4.5, 5.3, 5.4 List product/service evaluated.


Page 23 of 31


SECTION 5 - PROCUREMENT

Figure 5B


SUPPLIER NAME, LOCATION AND DATE(S) PERFORMED

5.4 List the suppliers from Figure 5A including the location and date the audit/source verification was performed.

EVALUATION METHOD (APPENDIX B AUDIT, SOURCE ACTIVITY)

5.4 List the method used to evaluate the supplier.

SCOPE 5.4 List the scope of the audit/source verification.

AUDITORS 5.4 List personnel who performed the audit/source verification.

NUMBER OF DEFICIENCIES (OPEN/CLOSED)

5.4 Enter the number of deficiencies and their status (open or closed) (i.e., 2-0, 3-C, etc.).

CORRECTIVE ACTION VERIFICATION METHOD AND DATE

5.4 List the method used (audit, document review, etc.) to verify corrective action and date.


Page 24 of 31


SECTION 6 - FABRICATION/ASSEMBLY ACTIVITIES

Figure 6A


ITEM DESCRIPTION (NAME, PART NO., P.O./CONTRACT NO., ETC.)

6.2 List description of part (Name, Part Number, P.O. Number, etc.)

WORK DOCUMENT 6.2 Shop Work Order Number, Traveler Number, etc. of items listed.

WORK ACTIVITY 6.2 List activity observed (e.g. assembly, special process, etc.).

WORK ACTIVITY PROCEDURE AND REV./DATE

6.2 List the work activity procedure number and revision/date for the work activity observed.


Page 25 of 31


SECTION 6 – MATERIAL CONTROL, HANDLING, SHIPPING, AND STORAGE

Figure 6B


ITEM DESCRIPTION (NAME, PART NO., P.O./CONTRACT NO., ETC.)

6.4, 6.5 List description of part (Name, Part Number, P.O./Contract Number, etc.)

METHOD OF IDENTIFICATION AND TRACEABILITY

6.3, 6.4, 6.5 List the method used to identify the item (heat number, P.O./Contract Number, etc.)

INSPECTION STATUS 6.4, 6.5 Indicate the status of the item (i.e., awaiting inspection, on hold, discrepant, rejected, etc.)


Page 26 of 31


SECTION 7 - SPECIAL PROCESSES

Figure 7A


ITEM DESCRIPTION (NAME, P/N, S/N, MODEL NO.)

7.2 List description of item (P/N, S/N, Model Number)

PROCESS 7.2 Enter process observed.

PROCEDURE AND REV./DATE 7.2 List procedure number and revision/date.

PERSONNEL QUALIFICATION AND LEVEL

7.2 List person's name, the discipline certified to and the level of qualification such as MT- Level II, RT- Level III, etc.

PROCEDURE QUALIFICATION 7.2 List PQR, or other qualification record.

EQUIPMENT QUALIFICATION 7.2 List equipment in use including M&TE. If calibration is required, evaluate as part of Checklist Item 8.6.


Page 27 of 31


Figure 7B WELDER/WELD OPERATOR


NAME/STAMP 7.2 List the name/stamp numbers of personnel being assessed.

CERT. TYPE (PROCESS & POSITIONS)

7.2 List the welding process certified to and positions, equipment and certification date.

CODE QUALIFIED TO 7.2 List the code that the welder is qualified to (i.e., AWS, ASME, etc.)

WELD PROCESS SPECIFICATION (WPS) AND REV./DATE

7.2 List the WPS that the welder is qualified to including revision/date.

MAINTENANCE OF QUALIFICATION 7.2 List qualification history of process evaluated, i.e. list history dates, use narrative words that history was acceptable, etc.


Page 28 of 31


SECTION 8 – TESTS, INSPECTIONS, AND CALIBRATION

Figure 8


ITEM DESCRIPTION (NAME, P/N, S/N, ETC.)

8.2 Description name, part number, serial number, model number, etc. of item being inspected/tested.

TEST/INSPECTION ACTIVITY TYPE AND DATE

8.2 Enter test/inspection activity such as receiving, in-process, sampling, continuity, functional, final, etc. and date of test/inspection.

CONTROLLING TEST/INSPECTION DOCUMENT TITLE/NUMBER AND REV./DATE

8.2 Enter title/number and revision/date of controlling test/inspection document.

INSPECTOR/TESTER NAME/STAMP

8.2 Identify Inspector/Tester performing activity.

ID NUMBER OF M&TE USED CALIBRATION CURRENT (Yes/No)

8.2, 8.4 Record the ID number of the M&TE used. Indicate by Yes/No that the M&TE’s calibration was current at the time of use.

RESULTS SAT. OR UNSAT. 8.2, 8.4 Enter Sat. or Unsat. for Results. If Unsat., record NCR number if applicable.


Page 29 of 31


SAMPLE OF COMPLETED FIGURE 8

ITEM DESCRIPTION (NAME, P/N, S/N,

ETC.)

TEST/INSPECTION ACTIVITY TYPE AND

DATE

CONTROLLING TEST/INSPECTION

DOCUMENT TITLE/NUMBER AND

REV./DATE

INSPECTOR/ TESTER

NAME/STAMP

ID NUMBER OF M&TE

CALIBRATION CURRENT (Yes/No)

RESULTS SAT OR UNSAT. IF UNSAT., RECORD NCR NO.

IF APPLICABLE

*all *all *all *all *all *all

Pallet Seal O-Ring P/N LF-240535

Receipt Inspection 01/30/03

Work Instruction LJ-240, Rev.1

QC-001 DHT-007 Yes

Sat.

4” Gate Valve P/N 100732

Hydrostatic Test 02/14/03

Hydrostatic Test Procedure HTP-002,

Rev. 10

Tom Smith PG-035, PG-036 Yes

Sat.

Motor 250TCZ Type P

CGI Dedication 03/30/03

CGDP-005 dated 08/02/02

QC-002 HPT-001 Yes

Sat.

Actuator Model SB-0, S/N L58115

Visual Inspection 04/15/03

Procedure IP039, Rev. 2

Greg Jones N/A Unsat. NCR 03-012


Page 30 of 31


SECTION 12 – INTERNAL AUDITS Figure 12

INTERNAL AUDITS


AUDIT SCOPE AND DATE 12.2 List the scope of the audit and the date the audit was performed.

AUDITOR(S) 12.2 List personnel who performed the audit. NUMBER OF DEFICIENCIES (OPEN/CLOSED)

12.2 Enter the number of deficiencies and their status (open or closed) (i.e., 2-0, 3-C, etc.).

CORRECTIVE ACTION VERIFICATION METHOD AND DATE

12.2 List the method used (audit, document review, etc.) to verify corrective action and date.


Page 31 of 31


SECTION 14 - TRAINING/CERTIFICATION Figure 14


NAME, STAMP, AND JOB TITLE 14.2, 14.3 List the name, stamp number if applicable,

and job title of personnel being assessed. INDOCTRINATION AND TRAINING COMPLETED (Yes/No)

14.2 Indicate by Yes/No whether indoctrination and training have been completed.

QUALIFICATION/CERTIFICATION CERT. TYPE AND LEVEL

14.3 List the area of qualification or certification (i.e., Lead Auditor, Inspector, ASME Professional Engineer, etc.). List the discipline certified to and the level (i.e., Mechanical - Level I, II or III).

Audit Guideline

Documents

Transcript of Audit Guideline