Volume102,Number6,November–December1997 …...Volume102,Number6,November–December1997...

Volume 102, Number 6, November–December 1997Journal of Research of the National Institute of Standards and Technology

[J. Res. Natl. Inst. Stand. Technol. 102, 647 (1997)]

Uncertainty and DimensionalCalibrations

Volume 102 Number 6 November–December 1997

Ted Doiron andJohn Stoup

National Institute of Standardsand Technology,Gaithersburg, MD 20899-0001

The calculation of uncertainty for a mea-surement is an effort to set reasonablebounds for the measurement resultaccording to standardized rules. Sinceevery measurement produces only an esti-mate of the answer, the primary requisiteof an uncertainty statement is to inform thereader of how sure the writer is that theanswer is in a certain range. This reportexplains how we have implemented theserules for dimensional calibrations of nine

different types of gages: gage blocks,gage wires, ring gages, gage balls, round-ness standards, optical flats indexingtables, angle blocks, and sieves.

Key words: angle standards; calibration;dimensional metrology; gage blocks;gages; optical flats; uncertainty; uncer-tainty budget.

Accepted: August 18, 1997

1. Introduction

The calculation of uncertainty for a measurement isan effort to set reasonable bounds for the measurementresult according to standardized rules. Since everymeasurement produces only an estimate of the answer,the primary requisite of an uncertainty statement is toinform the reader of how sure the writer is that theanswer is in a certain range. Perhaps the best uncer-tainty statement ever written was the following fromDr. C. H. Meyers, reporting on his measurements of theheat capacity of ammonia:

“We think our reported value is good to1 part in 10 000: we are willing to bet our ownmoney at even odds that it is correct to 2 parts in10 000. Furthermore, if by any chance our valueis shown to be in error by more than 1 part in1000, we are prepared to eat the apparatus anddrink the ammonia.”

Unfortunately the statement did not get past the NBSEditorial Board and is only preserved anecdotally [1].The modern form of uncertainty statement preserves thestatistical nature of the estimate, but refrains from

uncomfortable personal promises. This is less interest-ing, but perhaps for the best.There are many “standard” methods of evaluating and

combining components of uncertainty. An internationaleffort to standardize uncertainty statements has resultedin an ISO document, “Guide to the Expression of Uncer-tainty in Measurement,” [2]. NIST endorses this methodand has adopted it for all NIST work, including calibra-tions, as explained in NIST Technical Note 1297,“Guidelines for Evaluating and Expressing the Uncer-tainty of NIST Measurement Results” [3]. This reportexplains how we have implemented these rules fordimensional calibrations of nine different types ofgages: gage blocks, gage wires, ring gages, gage balls,roundness standards, optical flats indexing tables, angleblocks, and sieves.

2. Classifying Sources of Uncertainty

Uncertainty sources are classified according to theevaluation method used. Type A uncertainties areevaluated statistically. The data used for these calcula-

647


tions can be from repetitive measurements of the workpiece, measurements of check standards, or a combina-tion of the two. The Engineering Metrology Groupcalibrations make extensive use of comparator methodsand check standards, and this data is the primary sourcefor our evaluations of the uncertainty involved in trans-ferring the length from master gages to the customergage. We also keep extensive records of our customers’calibration results that can be used as auxiliary data forcalibrations that do not use check standards.Uncertainties evaluated by any other method are

called Type B. For dimensional calibrations the majorsources of Type B uncertainties are thermometer cali-brations, thermal expansion coefficients of customergages, deformation corrections, index of refractioncorrections, and apparatus-specific sources.For many Type B evaluations we have used a “worst

case” argument of the form, “we have never seen effectX larger than Y, so we will estimate that X is representedby a rectangular distribution of half-width Y.” We thenuse the rules of NIST Technical Note 1297, paragraph4.6, to get a standard uncertainty (i.e., one standarddeviation estimate). It is always difficult to assess thereliability of an uncertainty analysis. When a metrolo-gist estimates the “worst case” of a possible errorcomponent, the value is dependent on the experience,knowledge, and optimism of the estimator. It is alsoknown that people, even experts, often do not make veryreliable estimates. Unfortunately, there is little literatureon how well experts estimate. Those which do exist arenot encouraging [4,5].In our calibrations we have tried to avoid using “worst

case” estimates for parameters that are the largest, ornear largest, sources of uncertainty. Thus if a “worstcase” estimate for an uncertainty source is large,calibration histories or auxiliary experiments are used toget a more reliable and statistically valid evaluation ofthe uncertainty.We begin with an explanation of how our uncertainty

evaluations are made. Following this general discussionwe present a number of detailed examples. The generaloutline of uncertainty sources which make up ourgeneric uncertainty budget is shown in Table 1.

3. The Generic Uncertainty Budget

In this section we shall discuss each component of thegeneric uncertainty budget. While our examples willfocus on NIST calibration, our discussion of uncertaintycomponents will be broader and includes some sugges-tions for industrial calibration labs where the very low

level of uncertainty needed for NIST calibrations isinappropriate.

3.1 Master Gage Calibration

Our calibrations of customer artifacts are nearly al-ways made by comparison to master gages calibrated byinterferometry. The uncertainty budgets for calibrationof these master gages obviously do not have this uncer-tainty component. We present one example of this typeof calibration, the interferometric calibration of gageblocks. Since most industry calibrations are made bycomparison methods, we have focused on these meth-ods in the hope that the discussion will be more relevantto our customers and aid in the preparation of theiruncertainty budgets.For most industry calibration labs the uncertainty

associated with the master gage is the reported uncer-tainty from the laboratory that calibrated the mastergage. If NIST is not the source of the master gagecalibrations it is the responsibility of the calibrationlaboratory to understand the uncertainty statements re-ported by their calibration source and convert them, ifnecessary, to the form specified in the ISO Guide.In some cases the higher echelon laboratory is ac-

credited for the calibration by the National VoluntaryLaboratory Accreditation Program (NVLAP) adminis-tered by NIST or some other equivalent accreditationagency. The uncertainty statements from these laborato-ries will have been approved and tested by the accredi-tation agency and may be used with reasonable assur-ance of their reliabilities.

Table 1. Uncertainty sources in NIST dimensional calibrations

1. Master Gage Calibration2. Long Term Reproducibility3. Thermal Expansion

a. Thermometer calibrationb. Coefficient of thermal expansionc. Thermal gradients (internal, gage-gage, gage-scale)

4. Elastic DeformationProbe contact deformation, compression of artifacts undertheir own weight

5. Scale CalibrationUncertainty of artifact standards, linearity, fit routineScale thermal expansion, index of refraction correction

6. Instrument GeometryAbbe offset and instrument geometry errorsScale and gage alignment (cosine errors, obliquity, …)Gage support geometry (anvil flatness, block flatness, …)

7. Artifact EffectsFlatness, parallelism, roundness, phase corrections onreflection

648


Calibration uncertainties from non-accredited labora-tories may or may not be reasonable, and some form ofassessment may be needed to substantiate, or evenmodify, the reported uncertainty. Assessment of alaboratory’s suppliers should be fully documented.If the master gage is calibrated in-house by intrinsic

methods, the reported uncertainty should be docu-mented like those in this report. A measurement assur-ance program should be maintained, including periodicmeasurements of check standards and interlaboratorycomparisons, for any absolute measurements made bya laboratory. The uncertainty budget will not have themaster gage uncertainty, but will have all of the remain-ing components. The first calibration discussed inPart 2, gage blocks measured by interferometry, is anexample of an uncertainty budget for an absolutecalibration. Further explanation of the measurementassurance procedures for NIST gage block calibrationsis available [6].

3.2 Long Term Reproducibility

Repeatability is a measure of the variability of multi-ple measurements of a quantity under the same condi-tions over a short period of time. It is a component ofuncertainty, but in many cases a fairly small component.It might be possible to list the changes in conditionswhich could cause measurement variation, such as oper-ator variation, thermal history of the artifact, electronicnoise in the detector, but to assign accurate quantitativeestimates to these causes is difficult. We will not discussrepeatability in this paper.What we would really like for our uncertainty budget

is a measure of the variability of the measurementcaused by all of the changes in the measurement condi-tions commonly found in our laboratory. The term usedfor the measure of this larger variability caused bythe changing conditions in our calibration system isreproducibility.The best method to determine reproducibility is to

compare repeated measurements over time of the sameartifact from either customer measurement histories orcheck standard data. For each dimensional calibrationwe use one or both methods to evaluate our long termreproducibility.We determine the reproducibility of absolute calibra-

tions, such as the dimensions of our master artifacts, byanalyzing the measurement history of each artifact. Forexample, a gage block is not used as a master until it ismeasured 10 times over a period of 3 years. This ensuresthat the block measurement history includes variationsfrom different operators, instruments, environmentalconditions, and thermometer and barometer calibra-tions. The historical data then reflects these sources in a

realistic and statistically valid way. The historical dataare fit to a straight line and the deviations from the bestfit line are used to calculate the standard deviation.The use of historical data (master gage, check stan-

dard, or customer gage) to represent the variability froma particular source is a recurrent theme in the examplepresented in this paper. In each case there are two con-ditions which need to be met:

First, the measurement history must sample thesources of variation in a realistic way. This is a par-ticular concern for check standard data. The checkstandards must be treated as much like a customergage as possible.

Second, the measurement history must containenough changes in the source of variability to give astatistically valid estimate of its effect. For example,the standard platinum resistance thermometer(SPRT) and barometers are recalibrated on a yearlybasis, and thus the measurement history must span anumber of years to sample the variability caused bythese sensor calibrations.

For most comparison measurements we use twoNIST artifacts, one as the master reference and the otheras a check standard. The customer’s gage and both NISTgages are measured two to six times (depending on thecalibration) and the lengths of the customer block andcheck standard are derived from a least-squares fit ofthe measurement data to an analytical model of themeasurement scheme [7]. The computer records themeasured difference in length between the two NISTgages for every calibration. At the end of each year thedata from all of the measurement stations are sorted bysize into a single history file. For each size, the datafrom the last few years is collected from the history files.A least-squares method is used to find the best-fit linefor the data, and the deviations from this line are used tocalculate the estimated standard deviation, s [8,9]. Thiss is used as the estimate of the reproducibility of thecomparison process.If one or both of the master artifacts are not stable, the

best fit line will have a non-zero slope. We replace theblock if the slope is more than a few nanometers peryear.There are some calibrations for which it is impractical

to have check standards, either for cost reasons or be-cause of the nature of the calibration. For example, wemeasure so few ring standards of any one size that wedo not have many master rings. A new gage block stackis prepared as a master gage for each ring calibration.We do, however, have several customers who send thesame rings for calibration regularly, and these data canbe used to calculate the reproducibility of our measure-ment process.

649


3.3 Thermal Expansion

All dimensions reported by NIST are the dimensions ofthe artifact at 20 8C. Since the gage being measured maynot be exactly at 20 8C, and all artifacts change dimen-sion with temperature change, there is some uncertaintyin the length due to the uncertainty in temperature. Wecorrect our measurements at temperature t using thefollowing equation:

DL = a (20 8C–t )L (1)

where L is the artifact length at celsius temperature t ,DL is the length correction, a is the coefficient of ther-mal expansion (CTE), and t is the artifact temperature.This equation leads to two sources of uncertainty in

the correction DL : one from the temperature standarduncertainty, u(t ), and the other from the CTE standarduncertainty, u(a ):

U 2(dL ) = [aL ? u(t )]2 + [L (20 8C–t )u (a )]2 . (2)

The first term represents the uncertainty due to thethermometer reading and calibration. We use a numberof different types of thermometers, depending on therequired measurement accuracy. Note that for compari-son measurements, if both gages are made of the samematerial (and thus the same nominal CTE), the correc-tion is the same for both gages, no matter what thetemperature uncertainty. For gages of different materi-als, the correction and uncertainty in the correction isproportional to the difference between the CTEs of thetwo materials.The second term represents the uncertainty due to our

limited knowledge of the real CTE for the gage. Thissource of uncertainty can be made arbitrarily small bymaking the measurements suitably close to 20 8C.Most comparison measurements rely on one ther-

mometer near or attached to one of the gages. For thiscase there is another source of uncertainty, the temper-ature difference between the two gages. Thus, there arethree major sources of uncertainty due to temperature.

a. The thermometer used to measure the tempera-ture of the gage has some uncertainty.

b. If the measurement is not made at exactly 20 8C,a thermal expansion correction must be madeusing an assumed thermal expansion coefficient.The uncertainty in this coefficient is a source ofuncertainty.

c. In comparison calibrations there can be a temper-ature difference between the master gage and thetest gage.

3.3.1 Thermometer Calibration We used twotypes of thermometers. For the highest accuracy weused thermocouples referenced to a calibrated long stemSPRT calibrated at NIST with an uncertainty (3 stan-dard deviation estimate) equivalent to 0.001 8C. We ownfour of these systems and have tested them against eachother in pairs and chains of three. The systems agree tobetter than 0.002 8C. Assuming a rectangular dis-tribution with a half-width of 0.002 8C, we get astandard uncertainty of 0.002 8C/Ï3 = 0/0012 8C.Thus u(t ) = 0.0012 8C for SPRT/thermocouple sys-tems.For less critical applications we use thermistor based

digital thermometers calibrated against the primaryplatinum resistors or a transfer platinum resistor. Thesethermistors have a least significant digit of 0.01 8C. Ourcalibration history shows that the thermistors driftslowly with time, but the calibration is never in error bymore than 60.02 8C. Therefore we assume a rectangu-lar distribution of half-width of 0.02 8C, and obtainu(t ) = 0.02 8C/Ï3 = 0.012 8C for the thermistor sys-tems.In practice, however, things are more complicated. In

the cases where the thermistor is mounted on the gagethere are still gradients within the gage. For absolutemeasurements, such as gage block interferometry, weuse one thermometer for each 100 mm of gage length.The average of these readings is taken as the gage tem-perature.

3.3.2 Coefficient of Thermal Expansion (CTE) Theuncertainty associated with the coefficient of thermalexpansion depends on our knowledge of the individualartifact. Direct measurements of CTEs of the NIST steelmaster gage blocks make this source of uncertainty verysmall. This is not true for other NIST master artifactsand nearly all customer artifacts. The limits allowable inthe ANSI [19] gage block standard are 61310–6/8C.Until recently we have assumed that this was an ade-quate estimate of the uncertainty in the CTE. The vari-ation in CTEs for steel blocks, for our earlier measure-ments, is dependent on the length of the block. The CTEof hardened gage block steel is about 12310–6/8C andunhardened steel 10.5310–6/8C. Since only the ends oflong gage blocks are hardened, at some length the mid-dle of the block is unhardened steel. This mixture ofhardened and unhardened steel makes different parts ofthe block have different coefficients, so that the overallcoefficient becomes length dependent. Our previousstudies found that blocks up to 100 mm long were com-pletely hardened steel with CTEs near 12310–6/8C. TheCTE then became lower, proportional to the length over100 mm, until at 500 mm the coefficients were near10.5310–6/8C. All blocks we had measured in the pastfollowed this pattern.

650


Recently we have calibrated a long block set whichhad, for the 20 in block, a CTE of 12.6310–6/8C. Thisexperience has caused us to expand our worst case esti-mate of the variation in CTE from 61310–6/8C to62310–6/8C, at least for long steel blocks for which wehave no thermal expansion data. Taking 2310–6/8Cas the half-width of a rectangular distribution yieldsa standard uncertainty of u(a ) = (2310–6/8C)/Ï3= 1.2310–6 8C for long hardened steel blocks.For other materials such as chrome carbide, ceramic,

etc., there are no standards and the variability from themanufacturers nominal coefficient is unknown. Hand-book values for these materials vary by as much as1310–6/8C. Using this as the half-width of a rectangulardistribution yields a standard uncertainty ofu(a ) = ( 1310–6/8C)/Ï3 = 0.6310–6 8C for materialsother than steel.

3.3.3 Thermal Gradients For small gages thethermistor is mounted near the measured gage but on adifferent (similar) gage. For example, in gage blockcomparison measurements the thermometer is on a sep-arate block placed at the rear of the measurement anvil.There can be gradients between the thermistor and themeasured gage, and differences in temperature betweenthe master and customer gages. Estimating these effectsis difficult, but gradients of up to 0.03 8C have beenmeasured between master and test artifacts on nearly allof our measuring equipment. Assuming a rectangulardistribution with a half-width of 0.03 8C we obtaina standard uncertainty of u(Dt ) = 0.03 8C/Ï3= 0.017 8C. We will use this value except for specificcases studied experimentally.

3.4 Mechanical Deformation

All mechanical measurements involve contact ofsurfaces and all surfaces in contact are deformed. Insome cases the deformation is unwanted, in gage blockcomparisons for example, and we apply a correction toget the undeformed length. In other cases, particularlythread wires, the deformation under specified conditionsis part of the length definition and corrections may beneeded to include the proper deformation in the finalresult.The geometries of deformations occurring in our

calibrations include:1. Sphere in contact with a plane (for example,

gage blocks)2. Sphere in contact with an internal cylinder (for

example, plain ring gages)3. Cylinders with axes crossed at 908 (for exam-

ple, cylinders and wires)4. Cylinder in contact with a plane (for example,

cylinders and wires).

In comparison measurements, if both the master andcustomer gages are made of the same material, thedeformation is the same for both gages and there is noneed for deformation corrections. We now use two setsof master gage blocks for this reason. Two sets, one ofsteel and one of chrome carbide, allow us to measure95 % of our customer blocks without corrections fordeformation.At the other extreme, thread wires have very large

applied deformation corrections, up to 1 mm (40 min).Some of our master wires are measured according tostandard ANSI/ASME B1 [10] conditions, but many arenot. Those measured between plane contacts or betweenplane and cylinder contacts not consistent with the B1conditions require large corrections. When the masterwire diameter is given at B1 conditions (as is done atNIST), calibrations using comparison methods do notneed further deformation corrections.The equations from “Elastic Compression of Spheres

and Cylinders at Point and Line Contact,” by M. J.Puttock and E. G. Thwaite, [11] are used for all defor-mation corrections. These formulas require only theelastic modulus and Poisson’s ratio for each material,and provide deformation corrections for contacts ofplanes, spheres, and cylinders in any combination.The accuracy of the deformation corrections is as-

sessed in two ways. First, we have compared calcula-tions from Puttock and Thwaite with other publishedcalculations, particularly with NBS Technical Note 962,“Contact Deformation in Gage Block Comparisons”[12] and NBSIR 73-243, “On the Compression of aCylinder in Contact With a Plane Surface” [13]. In all ofthe cases considered the values from the differentworks were within 0.010 mm ( 0.4 min). Most of thisdifference is traceable to different assumptions aboutthe elastic modulus of “steel” made in the differentcalculations.The second method to assess the correction accuracy

is to make experimental tests of the formulas. A numberof tests have been performed with a micrometer devel-oped to measure wires. One micrometer anvil is flat andthe other a cylinder. This allows wire measurements ina configuration much like the defined conditions forthread wire diameter given in ANSI/ASME B1 ScrewThread Standard. The force exerted by the micrometeron the wire is variable, from less than 1 N to 10 N. Theforce gage, checked by loading with small calibratedmasses, has never been incorrect by more than a fewper cent. This level of error in force measurement isnegligible.The diameters measured at various forces were cor-

rected using calculated deformations from Puttock andThwaite. The deviations from a constant diameter arewell within the measurement scatter, implying that the

651

doiron

Typewritten Text

doiron

Typewritten Text


corrections from the formula are smaller than the mea-surement variability. This is consistent with the accuracyestimates obtained from comparisons reported in theliterature.For our estimate we assume that the calculated

corrections may be modeled by a rectangular distribu-tion with a half-width of 0.010 mm. The standard uncer-tainty is then u(def) = 0.010 mm/Ï3 = 0.006 mm.Long end standards can be measured either vertically

or horizontally. In the vertical orientation the standardwill be slightly shorter, compressed under its ownweight. The formula for the compression of a verticalcolumn of constant cross-section is

D(L ) =rgL 2

2E(3)

where L is the height of the column, E is the externalpressure, r is the density of the column, and g is theacceleration of gravity.This correction is less than 25 nm for end standards

under 500 mm. The relative uncertainties of the densityand elastic modulus of steel are only a few percent; theuncertainty in this correction is therefore negligible.

3.5 Scale Calibration

Since the meter is defined in terms of the speed oflight, and the practical access to that definition isthrough comparisons with the wavelength of light, alldimensional measurements ultimately are traceable toan interferometric measurement [14]. We use threetypes of scales for our measurements: electronic ormechanical transducers, static interferometry, anddisplacement interferometry.The electronic or mechanical transducers generally

have a very short range and are calibrated using artifactscalibrated by interferometry. The uncertainty of thesensor calibration depends on the uncertainty in theartifacts and the reproducibility of the sensor system.Several artifacts are used to provide calibration pointsthroughout the sensor range and a least-squares fit isused to determine linear calibration coefficients.

The main forms of interferometric calibration arestatic and dynamic interferometry. Distance is measuredby reading static fringe fractions in an interferometer(e.g., gage blocks). Displacement is measured by ana-lyzing the change in the fringes (fringe counting dis-placement interferometer). The major sources ofuncertainty—those affecting the actual wavelength—are the same for both methods. The uncertainties relatedto actual data readings and instrument geometry effects,however, depend strongly on the method and instru-ments used.The wavelength of light depends on the frequency,

which is generally very stable for light sources used formetrology, and the index of refraction of the medium thelight is traveling through. The wavelength, at standardconditions, is known with a relative standard uncertaintyof 1310–7 or smaller for most commonly used atomiclight sources (helium, cadmium, sodium, krypton).Several types of lasers have even smaller standard uncer-tainties—1310–10 for iodine stabilized HeNe lasers, forexample. For actual measurements we use secondarystabilized HeNe lasers with relative standard uncertain-ties of less than 1310–8 obtained by comparison to aprimary iodine stabilized laser. Thus the uncertaintyassociated with the frequency (or vacuum wavelength) isnegligible.For measurements made in air, however, our concern

is the uncertainty of the wavelength. If the index ofrefraction is measured directly by a refractometer, theuncertainty is obtained from an uncertainty analysis ofthe instrument. If not, we need to know the index ofrefraction of the air, which depends on the temperature,pressure, and the molecular content. The effect of eachof these variables is known and an equation to makecorrections has evolved over the last 100 years. Thecurrent equation, the Edlen equation, uses the tempera-ture, pressure, humidity and CO2 content of the air tocalculate the index of refraction needed to make wave-length corrections [15]. Table 2 shows the approximatesensitivities of this equation to changes in the environ-ment.

Table 2. Changes in environmental conditions that produce the indicated fractional changes in the wavelengthof light

Fractional change in wavelength

Environmental parameter 1310–6 1310–7 1310–8

Temperature 1 8C 0.1 8C 0.01 8CPressure 400 Pa 40 Pa 4 PaWater vapor pressure at 20 8C 2339 Pa 280 Pa 28 PaRelative humidity 100 %, saturated 12 % 1.2 %CO2 content (volume fraction in air) 0.006 9 0.000 69 0.000 069

652


Other gases affect the index of refraction in signifi-cant ways. Highly polarizable gases such as Freons andorganic solvents can have measurable effects at surpris-ingly low concentrations [16]. We avoid using solvents inany area where interferometric measurements are made.This includes measuring machines, such as micrometersand coordinate measuring machines, which usedisplacement interferometers as scales.Table 2 can be used to estimate the uncertainty in the

measurement for each of these sources. For example, ifthe air temperature in an interferometric measurementhas a standard uncertainty of 0.1 8C, the relative stan-dard uncertainty in the wavelength is 0.1310–6 mm/m.Note that the wavelength is very sensitive to air pressure:1.2 kPa to 4 kPa changes during a day, corresponding torelative changes in wavelength of 3310–6 to 10–5 arecommon. For high accuracy measurements the airpressure must be monitored almost continuously.

3.6 Instrument Geometry

Each instrument has a characteristic motion orgeometry that, if not perfect, will lead to errors. Thespecific uncertainty depends on the instrument, but thesources fall into a few broad categories: referencesurface geometry, alignment, and motion errors.Reference surface geometry includes the flatness and

parallelism of the anvils of micrometers used in ball andcylinder measurements, the roundness of the contacts ingage block and ring comparators, and the sphericity ofthe probe balls on coordinate measuring machines. Italso includes the flatness of reference flats used inmany interferometric measurements.

The alignment error is the angle difference and offsetof the measurement scale from the actual measurementline. Examples are the alignment of the two opposingheads of the gage block comparator, the laser or LVDTalignment with the motion axis of micrometers, and theillumination angle of interferometers.An instrument such as a micrometer or coordinate

measuring machine has a moving probe, and motion inany single direction has six degrees of freedom and thussix different error motions. The scale error is the errorin the motion direction. The straightness errors are themotions perpendicular to the motion direction. Theangular error motions are rotations about the axis ofmotion (roll) and directions perpendicular to the axis ofmotion (pitch and yaw). If the scale is not exactly alongthe measurement axis the angle errors produce measure-ment errors called Abbe errors.In Fig. 1 the measuring scale is not straight, giving

a pitch error. The size of the error depends onthe distance L of the measured point from the scaleand the angular error 1. For many instruments this Abbeoffset L is not near zero and significant errors canbe made.The geometry of gage block interferometers includes

two corrections that contribute to the measurement un-certainty. If the light source is larger than 1 mm in anydirection (a slit for example) a correction must be made.If the light path is not orthogonal to the surface of thegage there is also a correction related to cosine errorscalled obliquity correction. Comparison of results be-tween instruments with different geometries is an ade-quate check on the corrections supplied by the manufac-turer.

Fig. 1. The Abbe error is the product of the perpendicular distance of the scale from themeasuring point, L , times the sine of the pitch angle error, Q , error = L sinQ .

653


3.7 Artifact Effects

The last major sources of uncertainty are the proper-ties of the customer artifact. The most important ofthese are thermal and geometric. The thermal expansionof customer artifacts was discussed earlier (Sec. 3.3).Perhaps the most difficult source of uncertainty to

evaluate is the effect of the test gage geometry on thecalibration. We do not have time, and it is not economi-cally feasible, to check the detailed geometry of everyartifact we calibrate. Yet we know of many artifactgeometry flaws that can seriously affect a calibration.We test the diameter of gage balls by repeated com-

parisons with a master ball. Generally, the ball ismeasured in a random orientation each time. If the ballis not perfectly round the comparison measurementswill have an added source of variability as we sampledifferent diameters of the ball. If the master ball is notround it will also add to the variability. The checkstandard measurement samples this error in eachmeasurement.Gage wires can have significant taper, and if we

measure the wire at one point and the customer uses itat a different point our reported diameter will be wrongfor the customer’s measurement. It is difficult to esti-mate how much placement error a competent user of thewire would make, and thus difficult to include sucheffects in the uncertainty budget. We have made as-sumptions on the basis of how well we center the wiresby eye on our equipment.We calibrate nearly all customer gage blocks by

mechanical comparison to our master gage blocks. Thelength of a master gage block is determined by interfer-ometric measurements. The definition of length forgage blocks includes the wringing layer between theblock and the platen. When we make a mechanicalcomparison between our master block and a test blockwe are, in effect, assigning our wringing layer to the testblock. In the last 100 years there have been numerousstudies of the wringing layer that have shown that thethickness of the layer depends on the block and platenflatness, the surface finish, the type and amount of fluidbetween the surfaces, and even the time the block hasbeen wrung down. Unfortunately, there is still no way topredict the wringing layer thickness from auxiliarymeasurements. Later we will discuss how we haveanalyzed some of our master blocks to obtain a quantita-tive estimate of the variability.For interferometric measurements, such as gage

blocks, which involve light reflecting from a surface, wemust make a correction for the phase shift that occurs.There are several methods to measure this phase shift,all of which are time consuming. Our studies show that

the phase shift at a surface is reasonably consistent forany one manufacturer, material, and lapping process, sothat we can assign a “family” phase shift value to eachtype and source of gage blocks. The variability in eachfamily is assumed small. The phase shift for good qual-ity gage block surfaces generally corresponds to a lengthoffset of between zero (quartz and glass) and 60 nm(steel), and depends on both the materials and thesurface finish. Our standard uncertainty, from numerousstudies, is estimated to be less than 10 nm.Since these effects depend on the type of artifact, we

will postpone further discussion until we examine eachcalibration.

3.8 Calculation of Uncertainty

In calculating the uncertainty according to the ISOGuide [2] and NIST Technical Note 1297 [3], individualstandard uncertainty components are squared and addedtogether. The square root of this sum is the combinedstandard uncertainty. This standard uncertainty is thenmultiplied by a coverage factor k. At NIST this coveragefactor is chosen to be 2, representing a confidence levelof approximately 95 %.When length-dependent uncertainties of the form

a+bL are squared and then added, the square root is notof the form a+bL. For example, in one calibration thereare a number of length-dependent and length-indepen-dent terms:

u1 = 0.12 mm

u2 = 0.07 mm+0.03310–6 L

u3 = 0.08310–6 L

u4 = 0.23310–6 L

If we square each of these terms, sum them, and take thesquare root we get the lower curve in Fig. 2.Note that it is not a straight line. For convenience we

would like to preserve the form a+bL in our total uncer-tainty, we must choose a line to approximate this curve.In the discussions to follow we chose a length range andapproximate the uncertainty by taking the two endpoints on the calculated uncertainty curve and use thestraight line containing those points as the uncertainty.In this example, the uncertainty for the range0 to 1 length units would be the line f = a+bL containingthe points (0, 0.14 mm ) and (1, 0.28 mm).Using a coverage factor k = 2 we get an expanded

uncertainty U of U = 0.28 mm+0.28310–6 L for L be-tween 0 and 1. Most cases do not generate such a largecurvature and the overestimate of the uncertainty in themid-range is negligible.

654


3.9 Uncertainty Budgets for IndividualCalibrations

In the remaining sections we discuss the uncertaintybudgets of calibrations performed by the NIST Engi-neering Metrology Group. For each calibration we listand discuss the sources of uncertainty using the genericuncertainty budget as a guide. At the end of each discus-sion is a formal uncertainty budget with typical valuesand calculated total uncertainty.Note that we use a number of different calibration

methods for some types of artifacts. The method chosendepends on the requested accuracy, availability ofmaster standards, or equipment. We have chosen onemethod for each calibration listed below.Further, many calibrations have uncertainties that are

very sensitive to the size and condition of the artifact.The uncertainties shown are for “typical” customercalibrations. The uncertainty for any individual calibra-tion may differ considerably from the results in thiswork because of the quality of the customer gage orchanges in our procedures.The calibrations discussed are:

Gage blocks (interferometry)Gage blocks (mechanical comparison)Gage wires (thread and gear wires) andcylinders (plug gages)

Ring gages (diameter)Gage balls (diameter)Roundness standards (balls, rings, etc.)Optical flats Indexing tablesAngle blocksSieves

The calibration of line scales is discussed in a separatedocument [17].

4. Gage Blocks (Interferometry)

The NIST master gage blocks are calibrated by inter-ferometry using a calibrated HeNe laser as the lightsource [18]. The laser is calibrated against an iodine-stabilized HeNe laser. The frequency of stabilizedlasers has been measured by a number of researchersand the current consensus values of different stabilizedfrequencies are published by the International Bureau ofWeights and Measures [12]. Our secondary stabilizedlasers are calibrated against the iodine-stabilized laserusing a number of different frequencies.


This calibration does not use master reference gages.

Fig. 2. The standard uncertainty of a gage block as a function of length (a) and the linearapproximation (b).

655



The NIST master gage blocks are not used until theyhave been measured at least 10 times over a 3 year span.This is the minimum number of wrings we think willgive a reasonable estimate of the reproducibility andstability of the block. Nearly all of the current masterblocks have considerably more data than this minimum,with some steel blocks being measured more than50 times over the last 40 years. These data provide anexcellent estimate of reproducibility. In the long term,we have performed calibrations with many differenttechnicians, multiple calibrations of environmentalsensors, different light sources, and even different inter-ferometers.As expected, the reproducibility is strongly length

dependent, the major variability being caused bythermal properties of the blocks and measurementapparatus. The data do not, however, fall on a smoothline. The standard deviation data from our calibrationhistory is shown in Fig. 3.There are some blocks, particularly long blocks,

which seem to have more or less variability than thetrend would predict. These exceptions are usuallycaused by poor parallelism, flatness or surface finishof the blocks. Ignoring these exceptions the standarddeviation for each length follows the approximateformula:

u(rep) = 0.009 mm+0.08310–6 L (NIST Masters)

(4)

For interferometry on customer blocks the reproduci-bility is worse because there are fewer measurements.The numbers above represent the uncertainty of themean of 10 to 50 wrings of our master blocks. Customercalibrations are limited to 3 wrings because of time andfinancial constraints. The standard deviation of themean of nmeasurements is the standard deviation of thenmeasurements divided by the square root of n. We canrelate the standard deviation of the mean of 3 wrings tothe standard deviations from our master block historythrough the square root of the ratio of customer rings (3)to master block measurements (10 to 50). We will use 20as the average number of wrings for NIST masterblocks. The uncertainty of 3 wrings is then approxi-mately 2.5 times that for the NIST master blocks. Thestandard uncertainty for 3 wrings is

u(rep) = 0.022 mm+0.20310–6 L (3 wrings).

(5)


4.3.1 Thermometer Calibration The thermo-meters used for the calibrations have been changed overthe years and their history samples multiple calibrationsof each thermometer. Thus, the master block historicaldata already samples the variability from the thermome-ter calibration.Thermistor thermometers are used for the calibration

of customer blocks up to 100 mm in length. As dis-cussed earlier [(see eq. 2)] we will take the uncertainty

Fig. 3. Standard deviations for interferometric calibration of NIST master gage blocks of different length asobtained over a period of 25 years.

656


of the thermistor thermometers to be 0.01 8C. For longerblocks, a more accurate system consisting of a platinumSPRT (Standard Platinum Resistance Thermometer) asa reference and thermocouples is used.

4.3.2 Coefficient of Thermal Expansion (CTE)The CTE of each of our blocks over 25 mm in lengthhas been measured, leaving a very small standard uncer-tainty estimated to be 0.05310–6/8C. Since ourmeasurements are always within 60.1 8C of 20 8C, theuncertainty in length is taken to be 0.005310–6 L .

4.3.3 Thermal Gradients The long block tem-perature is measured every 100 mm, reducing theeffects of thermal gradients to a negligible level.The gradients between the thermometer and test

blocks in the short block interferometer (up to 100 mm)are small because the entire measurement space is in ametal enclosure. The gradients between the thermome-ter in the center of the platen and any block are less than0.005 8C. Assuming a rectangular distribution with ahalf-width 0.005 8C, we obtain a standard uncertainty of0.003 8C in temperature. For steel gage blocks(CTE = 11.5 mm/(m ? 8C) ), the standard uncertainty inlength is 0.003310–6 L . For other materials the uncer-tainty is less.

4.4 Elastic Deformation

We measure blocks oriented vertically, as specified inthe ANSI/ASME B89.1.9 Gage Block Standard [19].For customers who need the length of long blocks in thehorizontal orientation, a correction factor is used. Thiscorrection for self loading is proportional to the squareof the length, and is very small compared to othereffects. For 500 mm blocks the correction is only about25 nm, and the uncertainty depends on the uncertaintyin the elastic modulus of the gage block material. Nearlyall long blocks are made of steel, and the variationsin elastic modulus for gage block steels is only a fewpercent. The standard uncertainty in the correction isestimated to be less than 2 nm, a negligible addition tothe uncertainty budget.


The laser is calibrated against a well characterizediodine-stabilized laser. We estimate the relative standarduncertainty in the frequency from this calibration tobe less than 10–8, which is negligible for gage blockcalibrations.The Edlen equation for the index of refraction of air,

n, has a relative standard uncertainty of 3310–8.Customer calibrations are made under a single

environmental sensor calibration cycle and the uncer-tainty from these sources must be estimated. We checkour pressure sensors against a barometric pressure

standard maintained by the NIST Pressure Group.Multiple comparisons lead us to estimate the standarduncertainty of our pressure gages is 8 Pa. The airtemperature measurement has a standard uncertainty ofabout 0.015 8C, as discussed previously. By comparingseveral hygrometers we estimate that the standard uncer-tainty of the relative humidity is about 3 %.The gage block historical data contains measurements

made with a number of sources including elementaldischarge lamps (cadmium, helium, krypton) andseveral calibrated lasers. The historical data, therefore,contains an adequate sampling of the light sourcefrequency uncertainty.


The obliquity and slit corrections provided by themanufacturers are used for all of our interferometers.We have tested these corrections by measuring the sameblocks in all of the interferometers and have found nomeasurable discrepancies. Measuring blocks in interfer-ometers of different geometries could also be used tofind the corrections. For example, our Koesters typeinterferometer has no obliquity correction when prop-erly aligned, and the slit is accessible for measurement.Thus, the correction can be calculated. The Hilger inter-ferometer slits cannot be measured except by disassem-bly, but the corrections can be found by comparison ofmeasurements with the Koesters interferometer.The only geometry errors, other than those discussed

above, are due to the platen flatness. Each platen isexamined and is not used unless it is flat to 50 nm overthe entire 150 mm diameter. Since the gage block mea-surement is made over less than 25 mm of the surface,the local flatness is quite good. In addition, the measure-ment history of the master blocks has data from manyplatens and multiple positions on each platen, so thevariability from the platen flatness is sampled in thedata.

4.7 Artifact Geometry

The phase change that light undergoes on reflectiondepends on the surface finish and the electromagneticproperties of the block material. We assume that everyblock from a single manufacturer of the same materialhas the same surface finish and material, and thereforegives rise to the same phase change. We have restrictedour master blocks to a few manufacturers and materialsto reduce the work needed to characterize the phasechange. Samples of each material and manufacturer aremeasured by the slave block method [4], and theseresults are used for all blocks of similar material and thesame manufacturer.

657


In the slave block method, an auxiliary block, calledthe slave block, is used to help find the phase shiftdifference between a block and a platen. The methodconsists of two steps, shown schematically in Figs. 4and 5.

The interferometric length L test includes the mechani-cal length, the wringing film thickness, and the phasechange at each surface.Step 1. The test and slave blocks are wrung down to

the same platen and measured independently. The twolengths measured consist of the mechanical length of theblock, the wringing film, and the phase changes at thetop of the block and platen, as in Fig. 4.The general formula for the measured length of a

wrung block is:

Ltest = Lmechanical+Lwring+Lplaten phase–Lblock phase . (6)

For the test and slave blocks the formulas are

Ltest = Lt+Lt,w+(fplaten–ftest) (7)

Lslave = Ls+Ls,w+(fplaten–fslave) (8)

where Lt , Lt,w , Ls , and Ls,w are defined in Fig. 4.

Step 2. Either the slave block or both blocks are takenoff the platen, cleaned, and rewrung as a stack on theplaten. The length of the stack measured is:

Ltest+slave = Lt+Ls+Lt,w+Ls,w+(fplaten–fslave). (9)

If this result is subtracted from the sum of the twoprevious measurements, we find that

Ltest+slave–Ltest–Lslave = (ftest–fplaten). (10)

The weakness of this method is the uncertainty of themeasurements. The standard uncertainty of onemeasurement of a wrung gage block is about 0.030 mm(from the long term reproducibility of our master blockcalibrations). Since the phase measurement depends onthree measurements, the phase measurement has astandard uncertainty of about Ï3 times the uncertaintyof one measurement, or about 0.040 mm. Since thephase difference between block and platen is generallycorresponds to a length of about 0.020 mm, the un-certainty is larger than the effect. To reduce the uncer-tainty, a large number of measurements must be made,generally around 50. This is, of course, very timeconsuming.For our master blocks, using the average number of

slave block measurements gives an estimate of0.006 mm for the standard uncertainty due to the phasecorrection.We restrict our calibration service to small (8 to 10

block) audit sets for customers who do interferometry.These audit sets are used as checks on the customermeasurement process, and to assure that the uncertaintyis low we restrict the blocks to those from manufacturersfor which we have adequate phase-correction data. Theuncertainty is, therefore, the same as for our own masterblocks. On the rare occasions that we measure blocks ofunknown phase, the uncertainty is very dependent onthe procedure used, and is outside the scope of thispaper.If the gage block is not flat and parallel, the fringes

will be slightly curved and the position on the block

Fig. 4. Diagram showing the phase shift f on reflection makesthe light appear to have reflected from a surface slightly above thephysical metal surface.

Fig. 5. Schematic depiction of the measurements for determining thephase shift difference between a block and platen by the slave blockmethod.

658


where the fringe fraction is measured becomes impor-tant. For our measurements we attempt to read thefringe fraction as close to the gage point as possible.However, using just the eye, this is probably uncertain to1 mm to 2 mm. Since most blocks we measure are flatand parallel to 0.050 mm over the entire surface, theerror is small. If the block is 9 mmwide and the flatness/parallelism is 0.050 mm then a 1 mm error in the gagepoint produces a length error of about 0.005 mm. Forcustomer blocks this is reduced somewhat because threemeasurements are made, but since the readings aremade by the same person operator bias is possible. Weuse a standard uncertainty of 0.003 mm to account forthis possibility. Our master blocks are measured overmany years by different technicians and the variabilityfrom operator effects are sampled in the historical data.

5. Gage Blocks (Mechanical Comparison)

Most customer calibrations are made by mechanicalcomparison to master gage blocks calibrated on a regu-lar basis by interferometry. The comparison processcompares each gage block with two NIST master blocksof the same nominal size [20]. We have one steel and onechrome carbide master block for each standard size. Thecustomer block length is derived from the known lengthof the NIST master made of the same material toavoid problems associated with deformation corrections.

4.8 Summary

Tables 3 and 4 show the uncertainty budgets for inter-ferometric calibration of our master reference blocksand customer submitted blocks. Using a coverage factorof k = 2 we obtain the expanded uncertainty U of ourinterferometer gage block calibrations for our mastergage blocks as U = 0.022 mm+0.16310–6 L.The uncertainty budget for customer gage block

calibrations (three wrings) is only slightly different.The reproducibility uncertainty is larger because offewer measurements and because the thermal expansioncoefficient has not been measured on customer blocks.Using a coverage factor of k=2 we obtain an expandeduncertainty U for customer calibrations (three wrings)of U = 0.05 mm+0.4310–6 L .

Deformation corrections are needed for tungstencarbide blocks and we assign higher uncertainties thanthose described below.In the discussion below we group gage blocks into

three groups, each with slightly different uncertaintystatements. Sizes over 100 mm are measured on differ-ent instruments than those 100 mm or less, and havedifferent measurement procedures. Thus they form adistinct process and are handled separately. Blocksunder 1 mm are measured on the same equipment asthose between 1 mm and 100 mm, but the blocks have

Table 3. Uncertainty budget for NIST master gage blocks

Source of uncertainty Standard uncertainty (k = 1)

1. Master gage calibration N/A2. Long term reproducibility 0.009 mm+0.08310–6 L3. Thermometer calibration N/A4. CTE 0.005310–6 L5. Thermal gradients 0.030310–6 L up to L=0.1 m6. Elastic deformation Negligible7. Scale calibration 0.003310–6 L8. Instrument geometry Negligible9. Artifact geometry—phase correction 0.006 mm

Table 4. Uncertainty budget for NIST customer gage blocks measured by interferometry


1. Master gage calibration N/A2. Long term reproducibility 0.022 mm+0.2310–6 L3. Thermometer calibration N/A4. CTE 0.060310–6 L5. Thermal gradients 0.030310–6 L up to L=0.1 m6. Elastic deformation Negligible7. Scale calibration 0.003310–6 L8. Instrument geometry Negligible9. Artifact geometry—phase correction 0.006 mm10. Artifact geometry—gage point position 0.003 mm

659


different characteristics and are considered here as aseparate process. The major difference is that thinblocks are generally not very flat, and this leads to anextra uncertainty component. They are also so thin thatlength-dependent sources of uncertainty are negligible.


From the previous analysis (see Sec. 4.8) the standarduncertainty u of the length of the NIST master blocks isu = 0.011 mm+0.08310–6 L . Of course, some blockshave a longer measurement history than others, but forthis discussion we use the average. We use the actualvalue for each master block to calculate the uncertaintyreported for the customer block. Thus, numbers gener-ated in this discussion only approximate those in anactual report.


We use two NIST master gage blocks in everycalibration, one steel and the other chrome carbide.When the customer block is steel or ceramic, the steelblock length is the master (restraint in the data analysis).When the customer block is chrome or tungstencarbide, the chrome carbide block is the master. Thedifference between the two NIST blocks is a controlparameter (check standard).The check standard data are used to estimate the long

term reproducibility of the comparison process. Thetwo NIST blocks are of different materials so themeasurements have some variability due to contact forcevariations (deformation) and temperature variations(differential thermal expansion). Customercalibrations, which compare like materials, are lesssusceptible to these sources of variability. Thus, usingthe check standard data could produce an overestimateof the reproducibility. We do have some size rangeswhere both of the NIST master blocks are steel, and thevariability in these calibrations has been compared tothe variability among similar sizes where we havemasters of different material. We have found no sig-nificant difference, and thus consider our use of thecheck standard data as a valid estimate of the long termreproducibility of the system.The standard uncertainty derived from our control

data is, as expected, a smooth curve that rises slowlywith the length of the blocks. For mechanical compari-sons we pool the control data for similar sizes to obtainthe long term reproducibility. We justify this groupingby examining the sources of uncertainty. The inter-ferometry data are not grouped because the surfacefinish, material composition, flatness, and thermalproperties affect the measured length. The surface

finish and material composition affect the phase shiftand the flatness affects the wringing layer between theblock and platen. The mechanical comparisons are notaffected by any of these factors. The major remainingfactor is the thermal expansion. We therefore pool thecontrol data for similar size blocks. Each group hasabout 20 sizes, until the block lengths become greaterthan 25 mm. For these blocks the thermal differencesare very small. For longer blocks, the temperature ef-fects become dominant and each size represents aslightly different process; therefore the data are notcombined.For this analysis we break down the reproducibility

into three regimes: thin blocks (less than 1 mm), longblocks (>100 mm), and the intermediate range that con-tains most of the blocks we measure. This is a naturalbreakdown because blocks #100 mm are measuredwith a different type of comparator and a different com-parison scheme than are used for blocks >100 mm. A fitto the historical data produces an uncertainty com-ponent (standard deviation) for each group as shown inTable 5.


5.3.1 Thermometer Calibration For compari-son measurements of similar materials, the thermome-ter calibration is not very important since the tempera-ture error is the same for both blocks.

5.3.2 Coefficient of Thermal Expansion Thevariation in the CTE for similar gage block materials isgenerally smaller than the 61310–6 /8C allowed by theISO and ANSI gage block standards. From the variationof our own steel master blocks, we estimate the standarduncertainty of the CTE to be 0.4310–6/8C. Since we donot measure gage blocks if the temperature is more than0.2 8C from 20 8C, the length-standard uncertainty is0.08310–6 L . For long blocks (L>100 mm) we do notperform measurements if the temperature is more than0.1 8C from 20 8C, reducing the standard uncertainty to0.04310–6 L .

5.3.3 Thermal Gradients The uncertainty due tothermal gradients is important. For the short blockcomparator temperature differences up to 60.030 8Chave been measured between blocks positioned on the

Table 5. Standard uncertainty for length of NIST master gageblocks

Type of block Standard uncertainty

Thin (<1 mm) 0.008 mmIntermediate (1 mm to 100 mm) 0.004 mm+0.12310–6 LLong (>100 mm) 0.020 mm+0.03310–6 L

660


comparator platen. Assuming a rectangular distributionwe get a standard temperature uncertainty of 0.017 8C.The temperature difference affects the entire length ofthe block, and the length standard uncertainty is thetemperature difference times the CTE times the lengthof the block. Thus for steel it would be 0.20310–6 Land for chrome carbide 0.14310–6 L . For our simpli-fied discussion here we use the average value of0.17310–6 L .The precautions used for long block comparisons

result in much smaller temperature differences betweenblocks, 0.010 8C and less. Using this number as thehalf-width of a rectangular distribution we get astandard temperature uncertainty of 0.006 8C. Sincenearly all blocks over 100 mm are steel we find thestandard uncertainty component to be 0.07310–6 L .


Since most of our calibrations compare blocks of thesame material, the elastic deformation corrections arenot needed. There is, in theory, a small variability in theelastic modulus of blocks of the same material. We havenot made systematic measurements of this factor. Ourcurrent comparators have nearly flat contacts, fromwear, and we calculate that the total deformations areless than 0.05 mm. If we assume that the elastic proper-ties of gage blocks of the same material vary by less than5 % we get a standard uncertainty of 0.002 mm.We havetested ceramic blocks and found that the deformation isthe same as steel for our conditions.For materials other than steel, chrome carbide, and

ceramic (zirconia), we must make penetration cor-rections. Unfortunately, we have discovered that thediamond styli wear very quickly and the number ofmeasurements which can be made without measurablechanges in the contact geometry is unknown. From ourhistorical data, we know that after 5000 blocks, both ofour comparators had flat contacts. We currently add anextra component of uncertainty for measurementsof blocks for which we do not have master blocks ofmatching materials.


The gage block comparators are two point-contactdevices, the block being held up by an anvil. The lengthscale is provided by a calibrated linear variable differen-tial transformer (LVDT). The LVDT is calibrated insituusing a set of gage blocks. The blocks have nominallengths from 0.1 in to 0.100100 in with 0.000010 insteps. The blocks are placed between the contacts ofthe gage block comparator in a drift eliminatingsequence; a total of 44 measurements, four for eachblock, are made. The known differences in the lengthsof the blocks are compared with the measured voltages

and a least-squares fit is made to determine the slope(length/voltage) of the sensor. This calibration is doneweekly and the slope is recorded. The standard deviationof this slope history is taken as the standard uncertaintyof the sensor calibration, i.e., the variability of the scalemagnification. Over the last few years the relativestandard uncertainty has been approximately 0.6 %.Since the largest difference between the customer andmaster block is 0.4 mm (from customer histories), thestandard uncertainty due to the scale magnification is0.00630.4 mm = 0.0024 mm.The long block comparator has older electronics and

has larger variability in its scale calibration. This vari-ability is estimated to be 1 %. The long blocks also havea much greater range of values, particularly blocks man-ufactured before the redefinition of the in in 1959.When the in was redefined its value changed relative tothe old in by 2310–6, making the length value of allexisting blocks larger. The difference between our mas-ter blocks and customer blocks can be as large as 2 mm,and the relative standard uncertainty of 1 % in the scalelinearity yields a standard uncertainty of 0.020 mm.


If the measurements are comparisons between blockswith perfectly flat and parallel gaging surfaces, theuncertainties resulting from misalignment of thecontacts and anvil are negligible. Unfortunately, theartifacts are not perfect. The interaction of the surfaceflatness and the contact alignment is a small source ofvariability in the measurements, particularly for thinblocks. Thin blocks are often warped, and can be out offlat by 10 mm, or more. If the contacts are not alignedexactly or the contacts are not spherical, the contactpoints with the block will not be perpendicular to theblock. Thus the measurement will be slightly larger thanthe true thickness of the block. We have made multiplemeasurements on such blocks, rotating the block so thatthe angle between the block surface and the contact linevaries as much as possible. From these variations wefind that for thin blocks (<1 mm), the standard uncer-tainty is 0.010 mm.


The definition of length for a gage block is theperpendicular distance from the gage point on top to thecorresponding point on the flat surface (platen) to whichit is wrung. If the platen and gage block are perfectly flatthis distance would be the mechanical distance from thegage points on the top and bottom of the block plus thethickness of the wringing layer. If the customer blockalso was perfectly flat, the difference in the definedlength (from interferometry) and the mechanical length(from the two-point comparison) would be the same.

661


The customer block and the NIST master are not, ofcourse, perfectly flat. This leaves the possibility that thecalibration will be in error because the comparisonprocess, in effect, assigns the bottom geometry andwringing film of the NIST master to the customer block.We have attempted to estimate this error from our

history of the measurements of the 2 mm series ofmetric blocks. All of these blocks are steel and from thesame manufacturer, eliminating the complications ofthe interferometric phase correction. If there is no errordue to surface flatness, the length difference found byinterferometry and by mechanical comparisons shouldbe equal.Analyzing this data is difficult. Since either or both of

the blocks could be the cause of an offset, the averageoffset seen in the data is expected to be zero. Thesignature of the effect is a wider distribution of the datathan expected from the individual uncertainties in theinterferometry and comparison process.For each size the difference between interferometric

and mechanical length is a measure of the bias causedby the geometry of the gaging surfaces of the blocks.This bias is calculated from the formula

B = (L1int–L2int)–(L1mech–L2mech) (11)

where B is the bias, L1int and L2int are the lengths ofblocks 1 and 2 measured by interferometry, and L1mechand L2mech the lengths of blocks 1 and 2 measured bymechanical comparison. Because the geometry effectscan be of either sign, the average bias over a number ofblocks is zero. There is, fortunately, more useful infor-mation in the variation of the bias because it is made upof three components: the variations in the interfero-metric length, the mechanical length, and the geometryeffects. The variation in the interferometric andmechanical length differences can be obtained from theinterferometric history and the check standard data,respectively. Assuming that all of the distributions arenormal, the measured standard deviations are relatedby:

S2bias = S2

int = S2mech = S2

geom (12)

Our data for the 2 mm series is shown below. Thenumbers given are somewhat different than the tablesshow for typical calibrations for these sizes. The 2 mmseries is not very popular with our customers, and sincewe do few calibrations in these sizes there are fewerinterferometric measurements of the masters and fewercheck standard data. We analyzed 58 pairs of blocksfrom the 2 mm series blocks and obtained estimatedstandard deviations of 0.017 mm for the bias, 0.014 mmfor the interferometric differences and 0.005 mm for themechanical differences. This gives 0.008 mm as thestandard uncertainty in gage length due to the blocksurface geometry.Another way to estimate this effect is to measure the

blocks in two orientations, with each end wrung to theplaten in turn. We have not made a systematic study withthis method but we do have some data gathered in con-junction with international interlaboratory tests. Thisdata suggest that the effect is small for blocks under afew millimeters, but becomes larger for longer blocks.This suggests that the thin blocks deform to the shape ofthe plated when wrung, but longer blocks are stiffenough to resist the deformation. Since both of thesurfaces are made with the same lapping process, thisestimate may be somewhat smaller than the generalcase. This effect is potentially a major source of uncer-tainty and we plan further tests in the future.

5.8 Summary

The uncertainty budget for gage block calibration bymechanical comparison is shown in Table 6. Theexpanded uncertainty (coverage factor k = 2) for eachtype of calibration is

Thin Blocks (L<1mm) U=0.040mmGage Blocks (1mm to 100mm) U=0.030mm+0.35310–6 L

Long Blocks (100mm<L#500mm) U=0.055mm+0.20310–6L .

For long blocks with known thermal expansion coeffi-cients, the uncertainty is smaller than stated above.

Table 6. Uncertainty budget for NIST customer gage blocks measured by mechanical comparison


Thins (<1 mm) 1 mm to 100 mm over 100 mm

1. Master gage cal. 0.012 mm 0.012 mm+0.08310–6 L 0.012 mm+0.08310–6 L2. Reproducibility 0.008 mm 0.004 mm+0.12310–6 L 0.020 mm+0.03310–6 L3a. Thermometer cal. negligible negligible negligible3b. CTE 0.08310–6 L 0.08310–6 L 0.04310–6 L3c. Thermal Gradients 0.17310–6 L 0.17310–6 L 0.07310–6 L4. Elastic Deformation 0.002 mm 0.002 mm 0.002 mm5. Scale Calibration 0.002 mm 0.002 mm 0.020 mm6. Instrument Geometry 0.010 mm 0.002 mm 0.002 mm7. Artifact Geometry 0.008 mm 0.008 mm 0.008 mm

662


6. Gage Wires (Thread and Gear Wires)and Cylinders (Plug Gages)

Customer wires are calibrated by comparison tomaster wires using several different micrometers. Mostof the micrometers have flat parallel contacts, butcylinder contacts are occasionally used. The sensorsare mechanical twisted thread comparators, electronicLVDTs, and interferometers.For gage wires, gear wires, and cylinders we report

the undeformed diameter. If the wire is a thread wire theproper deformation is calculated and the corrected(deformed) diameter is reported.The master wires and cylinders have been calibrated

by a variety of methods over the last 20 years:

1. Large cylinders are usually calibrated by compari-son to gage blocks using a micrometer with flatcontacts.

2. A second device uses two optical flats with gageblocks wrung in the center as anvils. The upperflat has a fixture that allows it to be set at anyheight and adjusted nearly parallel to the bottomflat. The wire or cylinder is placed between thetwo gage blocks (wrung to the flats) and the topflat is adjusted to form a slight wedge. This wedgeforms a Fizeau interferometer and the distance be-tween the two flats is determined by multicolorinterferometry. The cylinder or wire diameter isthe distance between the flats minus the lengths ofthe gage blocks.

3. A third device consists of a moving flat anvil anda fixed cylindrical anvil. A displacement interfer-ometer measures the motion of the moving anvil.

4. Large diameter cylinders can be compared to gageblocks using a gage block comparator.

6.1 Master Artifact Calibration

The master wires are measured by a number ofmethods including interferometry and comparison togage blocks. We will take the uncertainty in the wiresand cylinders as the standard deviation of the mastercalibrations over the last 20 years. Because of thenumber of different measurement methods, each with itsown characteristic systematic errors, and the long periodof time involved, we assume that all of the pertinentuncertainty sources have been sampled. The standarddeviation derived from 168 degrees of freedom is0.065 mm.


We use check standards extensively in our wire calibra-tions, which produces a record of the long term repro-ducibility of the calibration. A typical data set is shownin Fig. 6.While we do not use check standards for every size

and type of wire, the difference in the measurementprocess for similar sizes is negligible. From ourlong-term measurement data we find the standarduncertainty for reproducibility (one standard deviation,300 degrees of freedom) is u = 0.025 mm.


6.3.1 Thermometer Calibration Since all cus-tomer calibrations are done by mechanical comparisonthe uncertainty due to the thermometer calibration isnegligible.

6.3.2 Coefficient of Thermal Expansion Nearlyall wires are steel, although some cylinders are made ofother materials. Since we measure within 0.2 8C of20 8C, if we assume a rectangular distribution and wewe know the CTE to about 10 %, we get a differentialexpansion uncertainty of 0.01310–6 L .

6.3.3 Thermal Gradients We have found tem-perature differences up to 60.030 8C in the calibrationarea of our comparators, and using 0.030 8C as the half-width of a rectangular distribution we get a standardtemperature uncertainty of 0.017 8C, which leads to alength standard uncertainty of 0.017310–6 L .


The elastic deformations under the measurement con-ditions called out in the screw thread standard [10] arevery large, 0.5 mm to 1 mm. Because we do not performmaster wire calibrations at standard conditions we mustmake corrections for the actual deformation during themeasurement to get the undeformed diameter, and thenapply a further correction to obtain the diameter at thestandard conditions. When both deformation correctionsare applied to the master wire diameter, the comparatorprocess automatically yields the correct standard diame-ter of the customer wires.Our corrections are calculated according to formulas

derived at Commonwealth Scientific and Industrial Re-search Organization (CSIRO), and have been checkedexperimentally. There is no measurable bias between thecalculated and measured deformations when the elasticmodulus of the material is well known. Unfortunately,there is a significant variation in the reported elastic

663


modulus for most common gage materials. An examina-tion of a number of handbooks for the elastic moduligives a relative standard deviation of 3 % for hardenedsteel, and 5 % for tungsten carbide.If we examine a typical case for thread wires (40

pitch) we have the corrections shown in Fig. 7. Linecontacts have small deformations and point contactshave large deformations. For a typical wire calibrationthe deformation at the micrometer zero, Dz, is a linecontact with a deformation of 0.003 mm. The deforma-tion of the wire at the micrometer flat contact , Dw /s,is also a line contact with a value of 0.003 mm. Thecontact between the micrometer cylinder anvil and wireis a point contact, Dw /a, which has the much largerdeformation of 0.800 mm.

Once these corrections are made the wire measure-ment is the undeformed diameter. To bring the reporteddiameter to the defined diameter (deformed at ASMEB1 conditions) a further correction of 1.6 mm must be

made. The corrections are thus from a slightly deformeddiameter, as measured, to the undeformed diameter, andfrom the undeformed diameter to the standard (B1)deformed diameter. Since all of the corrections use thesame formula, which we have assumed is correct, theonly uncertainty is in the difference between the twocorrections. In our example this is 0.8 mm.Nearly all gage wires are made of steel. If we take the

elastic modulus distribution of steel to be rectangularwith a half-width of 3 % and apply it to this differentialcorrection, we get a standard uncertainty of 0.013 mm.


The comparator scales are calibrated with gageblocks. Since several different comparators are used,each calibrated independently with different gageblocks, the check standard data adequately samples thevariability in sensor calibration.


When wires and cylinders are calibrated by compari-son, the instrument geometry is the same for both mea-surements and thus any systematic effects are the same.Since the difference between the measurements is usedfor the calibration, the effects cancel.


If the cylinder is not perfectly round, each time adifferent diameter is measured the answer will be differ-ent. We measure each wire or cylinder multiple times,changing the orientation each time. If the geometry of

Fig. 6. Check standard data for seven calibrations of a set of thread wires.

Fig. 7. Schematic depiction of the measurement of the wire/spindleand spindle/anvil contacts are line contacts and involve small defor-mations. The wire/anvil contact is a point contact and the deformationis large.

664


the wire is very bad this variation in the readingswill cause the calibration to fail the control test forrepeatability. If not, the wire will pass and the uncer-tainty assigned will be from the check standard data,i.e., the check standard wires. Since we pool checkstandard data from a number of similar size wires, thecheck standard data includes the effects of “average”roundness.If the customer wire is significantly more out of

round than our check standard the calibration will failthe repeatability test. In these cases we increase thereported uncertainty. For customers who need thehighest accuracy, we measure the roundness as part ofthe calibration or make measurements only along onemarked diameter. The customer then makes measure-ments using the same diameter.Wires and cylinders can also be tapered. Since we

measure the wires in the middle, but fixture the wires byhand, there is some uncertainty in the position of themeasurement. According to the thread wire standard,wires must be tapered less than 0.254 mm (10 min) overthe central 25 mm (1 in) of their length. Most of thewires we calibrate are master wires and are much betterthan this limit. Since the fixturing error is less than afew millimeters, the resultant uncertainty in diameter issmall. As an estimate we assume the central 25 mm ofthe wire has a possible diameter change of 0.1 mm,giving a possible diameter change of 0.008 mm for anassumed 2 mm fixturing shift.

6.8 Summary

Table 7 shows the uncertainty budget for wire andcylinder calibrations. We combine the standard un-certainties and use a coverage factor of k = 2 to obtainthe expanded uncertainty U for wires up to 25 mmin diameter of U = 0.14 mm+0.03310–6 L .The cylinder calibrations are done with little or no

deformation and therefore the last uncertainty source,elastic deformation, is negligible. The change is not,however, large enough to change the uncertainty in thesecond decimal digit.

7. Ring Gages (Diameter <100 mm)

Ring gages are measured by comparison to one ormore gage block stacks. Most ring gages have a markeddiameter and this particular diameter is the onlyreported value.


Ring gages, in general, do not come in sets ofstandardized sizes. Because of this we cannot havemaster ring gages. Instead we calibrate ring gages bycomparison to a stack of gage blocks wrung to a preci-sion square. A gage block stack is prepared the samelength as the nominal ring diameter. This stack is wrungto a precision square and a 5 mm block is wrung on topso that about 10 mm of the block extend beyond the endof the stack, as shown in Fig. 8. This extended surfaceof the top block and the surface of the square forms aninternal length to compare with the ring.

The square is longer and wider than the gage blockstack so that the fringe fractions between the surface ofthe square and the top of the gage block stack are clearlyvisible. The quality of the wring can be seen by examin-ing the fringes. If the fringes on the block stack andsquare are parallel and straight the wring is good.

Fig. 8. Schematic depiction of the use of a gage block stack for useas a master gage for ring gage calibration.

Table 7. Uncertainty budget for NIST customer gage wires and cylinders measured by mechanical comparison


1. Master gage calibration 0.065 mm2. Long term reproducibility 0.025 mm3a. Thermometer calibration Negligible3b. CTE 0.01310–6 L3c. Thermal gradients 0.017310–6 L4. Elastic deformation 0.013 mm5. Scale calibration N/A6. Instrument geometry N/A7. Artifact geometry 0.008 mm

665


Each stack is measured by multicolor interferometryto give the highest possible accuracy. The gap is thedifference between the measured length of the top blockand the length of the entire stack. Since the length of thetop block is measured on the square, there is no phasecorrection needed, reducing the uncertainty. As anestimate we use the uncertainty of customer blockcalibrations without the phase correction uncertainty.Since the gap is the difference of two measurementsthe total uncertainty is the root-sum-square of twomeasurements. The standard uncertainty is0.038 mm+0.2310–6 L .


The ring gages we calibrate come in a large variety ofsizes and it is impractical to have master ring gages andcheck standards. We do not calibrate enough rings ofany one size to generate statistically significant data. Asan alternative, we use data from our repeat customerswith enough independent measurements on the samegages to estimate our long term reproducibility.Measurements on four gages from one customer over thelast 20 years show a standard deviation of 0.025 mm.


7.3.1 Thermometer Calibration The ther-mometer calibration affects the length of the masterstack, but this effect has been included in the uncer-tainty of the stack (master).

7.3.2 Coefficient of Thermal Expansion Wechoose gage blocks of the same material as the customergage to make the master gage block stack. This reducesthe differential expansion coefficient. By using similarmaterials as test and master gage the standard uncer-tainty of the differential thermal expansion coefficient is0.6310–6/8C and all of the measurements are madewithin 0.2 8C of 20 8C. This uncertainty in thermalexpansion coefficient gives a length standard uncer-tainty of 0.230.6310–6 L , or 0.12310–6 L .

7.3.3 Thermal Gradients We have measured thetemperature variation of the ring gage comparatorand found it is generally less than 0.020 8C. Usingsteel as our example, the possible temperature dif-ference between gages produces a proportionalchange in the ring diameter DL /L of (11.5310–6)3(0.020 8C) = 0.23310–6. Since our reproducibilityincludes a number of measurements in different years,and thus different conditions, this component of uncer-tainty is sampled in the reproducibility data and is notconsidered as a separate component of uncertainty.


Since the master gage and ring are of the same mate-rial the elastic deformation corrections are nearly thesame. There is a small correction because in one casethe contact is a probe ball against a plane and in the ringcase the ball probe is against a cylinder. These correc-tions, however, are less than 0.050 mm. Since we makethe corrections, the only uncertainty is associated withour knowledge of the elastic modulus and Poisson’sRatio of the materials. Using 5 % as the relative stan-dard uncertainty of the elastic properties, we get a stan-dard uncertainty in the elastic deformation correction of0.005 mm.


The ring comparator is calibrated using two or morecalibrated gage blocks. Since the uncertainty of theseblocks is less than 0.030 mm, and the comparator scaleis 2.5 mm, the uncertainty in the slope is about 1 %(95 % confidence level). The difference between thering and gage block stack wrung as the master is lessthan 0.5 mm, leading to a standard uncertainty of 0.5 %of 0.5 mm, or 0.0025 mm.


The master and gage are manipulated to assure thatalignment errors are not significant. The ring is movedsmall amounts until the readings are maximized, and themaximum diameter is recorded. The gage block stack isrotated slowly to minimize its reading. Since both errorsare cosine errors this procedure is fairly simple. Anothererror is the squareness of the flat reference surface ofthe ring to its cylinder axis. This alignment is testedseparately and corrections are applied as needed.

The remaining source of error is the alignment of thecontacts. If the relative motion of the two contacts isparallel but not coincident, the transfer of length fromthe gage block stack (with flat parallel surfaces) to thering gage (cylindrical surface) will have an error whichis proportional to the square of the distance the twosensor axes are displaced. We have tested for this errorusing very small diameter cylinders and have found noeffect at the 0.025 mm level. This provides a bound onthe axis displacement of 5 mm. This level displacementwould produce possible errors in wring calibrations ofup to 0.020 mm on 3 mm rings and proportionatelysmaller errors on larger diameter rings. If we assume the0.020 mm represents the half-width of a rectangular dis-tribution, we get a standard uncertainty of 0.012 mm for3 mm rings. Since we rarely calibrate a ring with adiameter under 5 mm, we take 0.010 mm as our standarduncertainty.

666


7.7 Customer Artifact Geometry

Ring gages have a marked diameter and we measureonly this diameter. The roundness of the ring does notaffect the measurement. We do provide roundness tracesof the ring on customer request.

7.8 Summary

The uncertainty budget for ring gage calibrationis shown in Table 8. The expanded uncertaintyU for ring gages up to 100 mm diameter (k = 2) isU = 0.094 mm+0.36310–6 L .

8. Gage Balls (Diameter)

Gage balls are measured directly by interferometry orby comparison to master balls using a precisionmicrometer. The interferometric measurement is madeby having the ball act as the spacer between two coatedoptical flats or an optical flat and a steel platen. Theflats are fixtured so that they can be adjusted nearlyparallel, forming a wedge. The fringe fraction is read atthe center of the ball for each of four colors and ana-lyzed in the same manner as multi-color interferometryof gage blocks. A correction is applied for the deforma-tion of the flats in contact with the balls and when thesteel platen is used, and for the phase change of light onreflection from the platen.


Master balls are calibrated by interferometry, usingthe ball as a spacer in a Fizeau interferometer or bycomparison to gage blocks. The master ball historicaldata covers a number of calibration methods over thelast 30 years. An analysis of this data gives a standarddeviation of 0.040 mm with 240 degrees of freedom.Since these measurements span a number of differenttypes of sensors, multiple sensor calibrations, system-atic corrections, and environmental corrections, there

are very few sources of variation to list separately. Theonly significant remaining sources are the uncertaintiesof the frequencies of the cadmium spectra, which arenegligible for the typical balls (<30 mm) calibrated byinterferometry. We take the standard deviation of themeasurement history as the standard uncertainty of themaster balls.


The long term reproducibility of gage ball calibrationwas assessed by collecting customer data over the last 10years. The standard deviation, with 128 degrees of free-dom is found to be 0.035 mm. There is no evident lengthdependence because there are very few gage balls over30 mm in diameter. For large balls the uncertainty isderived from repeated measurements on the gage inquestion.


8.3.1 Thermometer Calibration Gage balls aremeasured by comparison to the master balls. Since ourmaster balls are steel, there is little uncertainty due tothe thermometer calibration for the calibration of steelballs. This is not true for other materials. Tungstencarbide is the worst case. For a thermometer calibrationstandard uncertainty of 0.01 8C, we get a standard un-certainty from the differential expansion of steel andtungsten carbide of 0.08310–6 L .

8.3.2 Coefficient of Thermal Expansion Wetake the relative standard uncertainty in the thermalexpansion coefficients of balls to be the same as for gageblocks, 10 %. Since our comparison measurements arealways within 0.2 8C of 20 8C the standard uncertaintyin length is 1310–6/ 8C30.2 8C3L = 0.2310–6 L .

8.3.3 Thermal Gradients We have found temper-ature differences up to 0.030 8C between balls, whichwould lead to a standard uncertainty of 0.3310–6 L .Using 60.030310–6 L as the span of a rectangular dis-tribution we get a standard uncertainty of 0.17310–6 L .

Table 8. Uncertainty budget for NIST customer gage blocks measured by mechanical comparison


1. Master gage calibration 0.038 mm+0.2310–6 L2. Long term reproducibility 0.025 mm3a. Thermometer calibration N/A3b. CTE 0.12310–6 L3c. Thermal gradients N/A4. Elastic deformation 0.005 mm5. Scale calibration 0.003 mm6. Instrument geometry 0.010 mm7. Artifact geometry Negligible

667



There are two sources of uncertainty due to elasticdeformation. The first is the correction applied whencalibrating the master ball. For balls up to 25 mm indiameter the corrections are small and the major sourceof uncertainty is from the uncertainty in the elasticmodulus. If we assume 5 % relative standard uncer-tainty in the elastic modulus, the standard uncertainty inthe deformation correction is 0.010 mm.The second source is from the comparison process. If

both the master and customer balls are of the samematerial, then no correction is needed and the uncer-tainty is negligible. If the master and customer balls areof different materials, we must calculate the differentialdeformation. The uncertainty of this correction is alsodue to uncertainty of the elastic modulus. While theuncertainty of the difference between the elastic proper-ties of the two balls is greater than for one ball, thedifferential correction is smaller than for the absolutecalibration of one ball, and the standard uncertaintyremains nearly the same, 0.010 mm.


The comparator scale is calibrated with a set of gageblocks of known length difference. Since the range ofthe comparator is 2 mm and the block lengths are knownto 0.030 mm, the slope is known to approximately 1 %.Customer blocks are seldom more than 0.3 mm from themaster ball diameter, so the uncertainty is less than0.003 mm.


The flat surfaces of the comparator are parallel tobetter than 0.030 mm. Since the balls are identicallyfixtured during the measurements, there is negligibleerror due to surface flatness. The alignment of the scalewith the micrometer motion produces a cosine error,which, given the very small motion, is negligible.


The reported diameter of a gage ball is the average ofseveral measurements of the ball in random orienta-tions. This means that if the customer ball is not veryround, the reproducibility of the measurement is de-graded. For customer gages suspected of large geome-try errors we will generally rotate the ball in themicrometer to find the range of diameters found. Insome cases roundness traces are performed. We adjustthe assigned uncertainty for balls that are significantlyout of round.

8.8 Summary

From Table 9 it is obvious that the length-dependentterms are too small to have a noticeable affect on thetotal uncertainty. For customer artifacts that are signifi-cantly out-of-round, the uncertainty will be largerbecause the reproducibility of the comparison isaffected. For these and other unusual calibrations, thestandard uncertainty is increased. The expanded uncer-tainty U (k = 2) for balls up to 30 mm in diameter isU = 0.11 mm.

9. Roundness Standards (Balls,Rings, etc.)

Roundness standards are calibrated on an instrumentbased on a very high accuracy spindle. A linear variabledifferential transformer (LVDT) is mounted on thespindle, and is rotated with the spindle while in contactwith the standard. The LVDT output is monitored by acomputer and the data is recorded. The part is rotated308 11 times and measured in each of the orientations.The data is then analyzed to yield the roundness ofthe standard as well as the spindle. The spindle round-ness is recorded and used as a check standard forthe calibration.

Table 9. Uncertainty budget for NIST customer gage balls measured by mechanical comparison


Uncertainty (general) Uncertainty (30 mm ball)

1. Master gage cal. 0.040 mm 0.040 mm2. Reproducibility 0.035 mm 0.035 mm3a. Thermometer cal. 0.08310–6 L 0.003 mm3b. CTE 0.20310–6 L 0.006 mm3c. Thermal Gradients 0.17310–6 L 0.005 mm4. Elastic Deformation 0.010 mm 0.010 mm5. Scale Calibration 0.003 mm 0.003 mm6. Instrument Geometry Negligible Negligible7. Artifact Geometry As needed As needed

668



The roundness calibration is made using a multiple-redundant closure method [21] and does not require amaster artifact.


Data from multiple calibrations of the same round-ness standards for customers were collected and ana-lyzed. The data included measurements of six differentroundness standards made over periods as long as 15years. The standard deviation of a radial measurement,derived from this historical data (60 degrees of free-dom), is 0.008 mm.


Measurements are made in a temperature controlledenvironment (60.1 8C) and care is taken to allow gradi-ents in the artifact caused by handling to equilibrate.The roundness of an artifacts is not affected by homoge-neous temperature changes of the magnitude allowedby our environmental control.


Since the elastic properties of the artifacts are homo-geneous the probe deformations are also homogeneousand thus irrelevant.

9.5 Sensor Calibration

The LVDT is calibrated with a magnificationstandard. At our normal magnification for roundnesscalibrations the magnification standard uncertainty isapproximately 0.10 mm over a 2 mm range. Sincemost roundness masters calibrated in our laboratoryhave deviations of less than 0.03 mm, the standarduncertainty due to the probe calibration is less than0.002 mm.


The closure method employed measures the geomet-rical errors of the instrument as well as the artifactand makes corrections. Thus only the non-reproduciblegeometry errors of the instrument are relevant, and theseare sampled in the multiple measurements and includedin the reproducibility standard deviation.


For roundness standards with a base, the squarenessof the base to the cylinder axis is important. If thisdeviates from 908 the cylinder trace will be an ellipse.Since the eccentricity of the trace is related to the cosineof the angular error, there is generally no problem. Ourroundness instrument has a Z motion (direction of thecylinder axis) of 100 mm and is straight to betterthan 0.1 mm. It is used to check the orientation of thestandard in cases where we suspect a problem.For sphere standards a marked diameter is usually

measured, or three separate diameters are measured andthe data reported. Thus there are no specific geometry-based uncertainties.

9.8 Summary

Table 10 gives the uncertainty budget for calibratingroundness standards. Since the thermal and scale uncer-tainties are negligible, the only major source of uncer-tainty is the long term reproducibility of the calibration.Using a coverage factor k = 1 the expanded uncertaintyU of roundness calibrations is U = 0.016 mm.

10. Optical Flats

Optical flats are calibrated by comparison to cali-brated master flats. The master flats are calibrated usingthe three-flat method, which is a self-calibrating method[22]. In the three flat method only one diameter iscalibrated. For our customer calibrations the test flat ismeasured and then rotated 908 so that a second diametercan be measured.

Table 10. Uncertainty budget for NIST customer roundness standards


1. Master gage calibration N/A2. Long term reproducibility 0.008 mm3a. Thermometer calibration N/A3b. CTE N/A3c. Thermal gradients N/A4. Elastic deformation N/A5. Scale calibration 0.002 mm6. Instrument geometry N/A7. Artifact geometry N/A

669


The test flat is placed on top of the master flat,supported by three thin spacers placed 0.7 times theradius from the center at 1208 angles from each other.The master flat is supported on a movable carriage in asimilar (three point) manner. These supports assure thatthe measured diameter of both flats are undeformedfrom their free state. For metal or partially coated refer-ence flats the test flat is place on the bottom and themaster flat placed on top.One of the three spacers between the flats is slightly

thicker than the other two, making the space betweenthe flats a wedge. When this wedge is illuminated bymonochromatic light, distinct fringes are seen. Thestraightness of these fringes corresponds to the distancebetween the flats, and is measured using a Pulfrichviewer [23].


The master flat is calibrated with the same apparatusused for customer calibrations, the only difference beingthat for a customer calibration the customer flat iscompared to a master flat, and for master flat calibra-tions, the master flat is compared with two other masterflats of similar size. Sources of uncertainty other thanthe long term reproducibility of the comparisonmeasurement are negligible (see Secs. 11.3 to 11.7).The actual three flat calibration of the master flat uses

comparisons of all three flats against each other in pairs.The contour is measured on the same diameter on eachflat for all of the combinations. The first measurementusing flats A and B is

mAB(x ) = FA(x )+FB(x ), (13)

where F (x ) is the variation in the height of the air layerbetween the two flats. The value is positive when thesurface is outside of the line connecting the endpoints(i.e., a convex flat has F (x ) positive everywhere). FlatC replaces flat B and the contour along the same diame-ter is remeasured:

mAC(x ) = FA(x )+FC(x) (14)

Flat B is placed on the bottom and C on top and thecontour is measured.

mBC(x ) = FB(x )+FC(x ). (15)

The shape of flat A is then

FA(x ) = 12 [mAB(x )+mAC(x )–mBC(x )] (16)

Since all three measurements use the same procedurethe uncertainties are the same. If we denote the standarduncertainty of one flat comparison as u, the standarduncertainty uA in FA(x ) is related to u by

uA = Î 3u 2

4 . (17)

Thus the standard uncertainty of the master flat is thesquare root of 3/4 or about 0.9 times the standard uncer-tainty of one comparison.To estimate the long term reproducibility, we have

compared calibrations of the same flat using two differ-ent master flats over an eight year period. This compari-son shows a standard deviation (60 degrees of freedom)of 3.0 nm. Using this value in Eq. (16) we find thestandard uncertainty of the master flat to be 0.0026 mm.


As noted above, for a customer flat the standard un-certainty of the comparison to the master flat is0.003 mm.


The geometry of optical flats is relatively unaffectedby small homogeneous temperature changes. Since thecalibrations are done in a temperature controlled envi-ronment (60.1 8C ), there is no correction or uncer-tainty related to temperature effects.


The flatness of the surface of an optical flat dependsstrongly on the way in which it is supported. Ourcalibration report includes a description of the supportpoints and the uncertainty quoted applies only when theflat is supported in this manner. Changing the supportpoints by small amounts (1 mm or less, characteristic ofhand placement of the spacers) produces negligiblechanges in surface flatness.


The basic scale of the measurement is the wavelengthof light. For optical flats the fringe straightness issmaller than the fringe spacing, and is measured toabout 1 % of the fringe spacing. Thus the wavelength ofthe light need only be known to better than 1 %. Sincea helium lamp is used for illumination, even if the indexof refraction corrections are ignored the wavelength isknown with an uncertainty that is a few orders ofmagnitude smaller than needed.

670



The flats are transported under the viewer on a onedimensional translation stage. Since the fringes are lessthan 5 mm apart and are measured to about 1 % of afringe spacing, as long as the straightness of the waybedmotion is less than 5 mm the geometry correction isnegligible. In fact, the waybed is considerably betterthan needed.


There are no test artifact-related uncertainty sources.

10.8 Summary

Table 11 shows the uncertainty budget for optical flatcalibration. The only non-negligible uncertainty sourceis the master flat and the comparison reproducibility.The expanded uncertainty U (k = 2) of the calibration istherefore U = 0.008 mm.

11. Indexing Tables

Indexing tables are calibrated by closure methods us-ing a NIST indexing table as the second element and acalibrated autocollimator as the reference [24]. The cus-tomer’s indexing table is mounted on a stack of twoNIST tables. A plane mirror is then mounted on top ofthe customer table. The second NIST table is not part ofthe calibration but is only used to conveniently rotate theentire stack.Generally tables are calibrated at 308 intervals. Both

indexing tables are set at zero and the autocollimatorzeroed on the mirror. The customer’s table is rotatedclockwise 308 and our table counter-clockwise 308. Thenew autocollimator reading is recorded. This procedureis repeated until both tables are again at zero.The stack of two tables is rotated 308, the mirror

repositioned, and the procedure repeated. The stack isrotated until it returns to its original position. Fromthe readings of the autocollimator the calibration ofboth the customer’s table and our table is obtained.The calibration of our table is a check standard forthe calibration.


As discussed above there is no master needed in aclosure calibration.


Each indexing calibration produces a measurementrepeatability for the procedure. Our normal calibrationuses the closure method, comparing the 308 intervals ofthe customer’s table with one of our tables. One of the308 intervals may be subdivided into six 58 subintervals,and one of the 58 subintervals may be subdivided into 18subintervals. The method of obtaining the standard devi-ation of the intervals is documented in NBSIR 75-750,“The Calibration of Indexing Tables by Subdivision,” byCharles Reeve [24]. Since each indexing table is differ-ent and may have different reproducibilities we use thedata from each calibration for the uncertainty evalua-tion.As an example and a check on the process, we have

examined the data from the repeated calibration of theNIST indexing table used in the calibration. Six calibra-tions over a 10 year span show a pooled standard devia-tion of 0.07'' for 308 intervals. The average uncertainty(based on short term repeatability of the closure proce-dure) for each of the calibrations is within round-off ofthis value, showing that the short and long term repro-ducibility of the calibration is the same.


The calibrations are performed in a controlled ther-mal environment, within 0.1 8C of 20 8C. Temperatureeffects on indexing tables in this environment are negli-gible.


There is no contact with the sensors so there is nodeformation caused by the sensor. There is deformationof the indexing table teeth each time the table is reposi-tioned. This effect is a major source of variability in themeasurement, and is adequately sampled in the proce-dure.

Table 11. Uncertainty budget for NIST customer optical flats


1. Master gage calibration 0.0026 mm2. Long term reproducibility 0.0030 mm3a. Thermometer calibration N/A3b. CTE N/A3c. Thermal gradients Negligible4. Elastic deformation Negligible5. Scale calibration Negligible6. Instrument geometry Negligible7. Artifact geometry N/A

671



The autocollimators are calibrated in a variety ofways, including differential motions of stacked index-ing tables, reversal of angle blocks (typically 1'' and5'' ), precision angle generators, sine plates and com-parison with commercial laser interferometer based an-gle measurement systems. The uncertainty in generat-ing a 10'' angle change by any of these methods is small.Since the high quality indexing tables calibrated at

NIST typically have deviations from nominal of lessthan 2'' , the uncertainty component related the autocol-limator calibration is negligible on the order of 0.01'' ,which is negligible.


There are several subtle problems due to the flatnessof the reference mirror and alignment of the two index-ing tables that affect the calibration. However, withproper alignment of the table and mirror, the autocolli-mator will illuminate the same area of the mirror foreach measurement. This eliminates the effects of themirror flatness on the measurement.


The rotational errors (runout, tilt) of the typicalindexing table are too small to have a measurable effecton the measurement.

11.8 Summary

Table 12 shows the uncertainty budget for indexingtable calibrations. The expanded uncertainty U (k = 2)of indexing table calibrations is estimated to beU = 0.14'' .

12. Angle Blocks

Angle blocks are calibrated by comparison to masterangle blocks using an angle block comparator. Theangle block comparator consists of two high accuracyautocollimators and a fixture which allows angle

blocks of the same size to be positioned repeatably in themeasurement paths of the autocollimators. The autocol-limators are adjusted to zero on the surfaces of themaster angle block, and then the customer angle block issubstituted for the master. Customer angle blocks, themaster angle block, and a check standard are eachmeasured multiple times. The changes in the auto-collimator readings are recorded and analyzed to yieldthe angles of the customer blocks, the angle of the checkstandard, and the standard deviation of the comparisonscheme. The latter two items of data are used as statisti-cal process control parameters.


The master angle blocks are measured by a number ofmethods depending on their angle. Angle blocks ofnominal angle 1' or less can be calibrated using anindexing table and autocollimator by simple reversal.The 158 and larger blocks are calibrated by closuremethods related to the indexing table calibration. Inthese methods the angle of the angle block is comparedwith similar angles of the indexing table. For example,a 908 angle block is compared to the 08–908, 908–1808,1808–2708, and 2708–08 intervals of the indexing table.Using the known sum of the angles (3608) as therestraint for a least squares fit of the data, the angle ofthe block can be calculated. Note that there is no un-certainty in the restraint. The blocks between theseextremes are more of a challenge.The smaller angles are compared to subdivisions of a

calibrated indexing table. For example, the 58 angleblock is compared to each of the 58 subdivisions of aknown 308 interval of a calibrated table. The calibratedvalue of this 308 interval is used as the restraint. Sincewe are not doing a 3608 closure, this restraint does nothave zero uncertainty. The 308 uncertainty is, however,apportioned to each of the six subdivisions, therebyreducing its importance in our final calculations. Thusthe uncertainty from this calibration is not expected tobe significantly higher than the full closure method.

Table 12. Uncertainty budget for NIST customer indexing tables


1. Master gage calibration N/A2. Long term reproducibility 0.07''3a. Thermometer calibration N/A3b. CTE N/A3c. Thermal gradients N/A4. Elastic deformation N/A5. Scale calibration 0.01''6. Instrument geometry N/A7. Artifact geometry N/A

672


To assess the reproducibility of the calibration we ana-lyze the calibration history of our master blocks. Frommeasurements caried out over a 30 year period, we findthe standard devaition to be 0.073'' (213 degrees offreedom). There is no apparent dependence on the sizeof the angle.


Customer angle blocks are calibrated by comparisonto the master angle blocks using two autocollimators setup so that each autocollimator is at null on a face of themaster block [25]. The customer block is then put in theplace of the master block and the two autocollimatorreadings are recorded. The scheme used is a drifteliminating design with two NIST master blocks used toprovide both the restraint (sum of angles) and control(difference between angles) for the calibration. Weestimate the reproducibility of the measurement fromthese control measurements.Analysis of check standard data from calibrations

performed over the last 10 years yields a standard devi-ation of 0.059'' (380 degrees of freedom).Another check is to examine our customer historical

data. Figure 9 shows a small part of that history: ninecalibrations of one set of angle blocks over a 20 yearperiod.


Angle blocks are robust against angle changes causedby small homogeneous temperature changes. Tongs andgloves are used when handling the blocks to prevent

temperature gradients that would cause angle errors.The blocks are measured in a small box and allowed tocome to equilibrium before the data is taken, furtherreducing possible temperature effects. Any residualeffects are sampled in the control history and are notlisted separately.


There is no mechanical contact.


The uncertainty in the sensor (autocollimator) is thesame as described in the earlier discussion of indexingtables.


The only instrument geometry error arises if theangle block surface is not perpendicular to the auto-collimator axis in the nonmeasuring direction. Thiserror is a cosine error and is negligible in our setup.


Since the angle blocks are not exactly flat, it is possi-ble that the surface area illuminated during the NISTcalibration will not be the same area used by the cus-tomer. Since this is dependent on the customer’s equip-ment we do not include this source in our uncertaintybudget. The possibility of errors arising from the use ofthe angle block in a manner different from our calibra-tion is indicated in our calibration report.

Fig. 9. The variation of 16 gage blocks for 9 calibrations over 20 years. Each point is the measured deviationof a block from its historical mean calculated from the 9 calibrations.

673


12.8 Summary

Table 13 shows the uncertainty budget for angle blockcalibrations. The expanded uncertainty U (k = 2) isU = 0.18''.

13. Sieves

The Dimensional Metrology Group certifies wiremesh testing sieves to the current revision of ASTMSpecification E-11[26]. We test the average wire dia-meter, average hole diameter, and the frame and skirtdiameter. The frame and skirt diameters are checkedwith GO and NOGO gages, and therefore do not have anassociated uncertainty.Wire and hole diameters are measured with a cali-

brated optical projector. Hole diameters are measuredindirectly; the pitch of the sieve is measured and themeasured average wire diameter is subtracted to give theaverage hole size.The uncertainty of the pitch (number of wires per

centimeter) is very small. The sieve is mounted on anoptical projector or traveling microscope. The sieve ismoved until 100 wires have passed an index mark, andthe pitch is calculated. For number 5 to number 50sieves the number of wires is counted over a distance of100 mm. The standard uncertainty of the measuringscale is less than 10 mm over any 100 mm of travel,giving a standard relative pitch uncertainty of 0.01 %.This is considerably smaller than the standard uncer-tainty of the wire diameter measurement and is ignored.


Sieves are measured directly, so there are no masterartifacts.


We do not have check standards for sieve cali-brations. We have, however, made multiple measure-ments on sieves using a number of different measuringmethods.

For determining the pitch (average wire spacing) wehave used different Moire scales, a traveling microme-ter, and an optical projector to measure a single sieve.We find that the different methods all agree to within0.5 mm or better for every sieve examined.Measuring wire diameter optically is difficult be-

cause of diffraction effects at the edges of the wire. Thediameter varies quite widely depending on the type oflighting (direction, coherence) and the quality of theoptics. We have compared a number of differentmethods using back lighting, front lighting, diffuse andcollimated light, and different optical systems. For thesemeasurements both stage micrometers and calibratedwires have been used to calibrate the sensors. We findthat these results agree within 2 mm. Having no cleartheoretical reason to choose one method over the other,we take this spread as the uncertainty of opticalmethods. Taking the value of 2 mm as the half widthof a rectangular distribution, we estimate the standarduncertainty to be 1 mm.


The temperature control of our laboratory is adequateto make the uncertainty due to thermal effects negligiblewhen compared to the tolerances required by the ASTMspecification.


There is no mechanical contact.


The optical projector is calibrated with a precisionstage micrometer. The stage micrometer has beencalibrated at NIST and has a standard uncertainty of lessthan 0.03 mm. Since the optical comparator has a leastcount of only 1 mm, the stage micrometer length uncer-tainty is negligible. The uncertainty of the opticalprojector scale is taken as a rectangular distribution withhalf-width of 0.5 mm, giving an standard uncertainty of0.29 um.

Table 13. Uncertainty budget for NIST customer angle blocks


1. Master gage calibration 0.075''2. Long term reproducibility 0.060''3a. Thermometer calibration N/A3b. CTE N/A3c. Thermal gradients N/A4. Elastic deformation N/A5. Scale calibration 0.010''6. Instrument geometry N/A7. Artifact geometry N/A

674


The correlation tests described earlier provide a prac-tical test of the accuracy of the scale calibration.


The major source of instrument uncertainty is thepitch error of the optical projector and traveling micro-scopes. Since both have large Abbe offsets the errors areas large as 20 mm. However, for fine sieves with toler-ances of 3 mm to 10 mm, at least 300 wire spacings aremeasured to get the average pitch. For larger sieves,fewer wire spacings are measured but the tolerances arelarger. In all cases the resulting error is far below thetolerance, and is ignored.


Customer sieves that have flatness problems arerejected as unmeasurable.

13.8 Summary

The major tests of sieves are the average wire and holediameter. Since we calculate the hole diameter from thewire diameter and average wire spacing, the only non-negligible uncertainty is from the wire diameter mea-surement. Our experiments show that the variation be-tween methods is much larger than the reproducibilityof any one method. This variation between methods(two standard deviations, 95 % confidence) is taken asthe expanded uncertainty U = 2 mm.

Acknowledgment

The authors would like to thank the metrologists, atNIST and in industry, who were kind enough to read andcomment on drafts of this paper. We would like to thank,particularly, Ralph Veale and Clayton Teague of thePrecision Engineering Division, who made many valu-able suggestions regarding the content and presentationof this work.

14. References

[1] Round-Table Discussion on Statement of Data and Errors,Nuclear Instrum. and Methods 112, 391 (1973).

[2] Guide to the Expression of Uncertainty in Measurement, Inter-national Organization for Standardization, Geneva, Switzerland(1993).

[3] B. N. Taylor and C. E. Kuyatt, Guidelines for Evaluating andExpressing the Uncertainty of NIST Measurement Results,Technical Note 1297, 1994 Edition, National Institute of Stan-dards and Technology (1994).

[4] Roger M. Cook, Experts in Uncertainty, Oxford University Press(1991).

[5] Massimo Piattelli-Palmarini, Inevitable Illusions, JohnWiley & Sons, Inc. (1994).

[6] Ted Doiron and John Beers, The Gage Block Handbook, Mono-graph 180, National Institute of Standards and Technology(1995).

[7] Carroll Croarkin, Measurement Assurance Programs. Part II:Development and Implementation, Special Publication 676-II,National Institute of Standards and Technology (1985).

[8] Ted Doiron, Drift Eliminating Designs for Non-SimultaneousComparison Calibrations, J. Res. Natl. Inst. Stand. Technol.98(2), 217-224 (1993).

[9] J. M. Cameron, The Use of the Method of Least Squares inCalibration, National Bureau of Standards Interagency Report74-587, National Bureau of Standards (U.S.) (1974).

[10] Gages and Gaging for Unified Inch Screw Threads, ANSI/ASME B1.2-1983, The American Society of Mechanical Engi-neers, New York, N.Y. (1983).

[11] M. J. Puttock and E. G. Thwaite, Elastic Compression ofSpheres and Cylinders at Point and Line Contact, NationalStandards Laboratory Technical Paper No. 25, CommonwealthScientific and Industrial Research Organization (1969).

[12] John S. Beers and James E. Taylor, Contact Deformation in GageBlock Comparisons, Technical Note 962, National Bureau ofStandards (U.S.) (1978).

[13] B. Nelson Norden, On the Compression of a Cylinder in ContactWith a Plane Surface, Interagency Report 73-243, NationalBureau of Standards (U.S.) (1973).

[14] Documents Concerning the New Definition of the Metre,Metrologia 19, 163-177 (1984).

[15] K. P. Birch and M. J. Downs, An Updated Equation for theRefractive Index of Air, Metrologia 30, 155-162 (1993).

[16] K. P. Birch, F. Reinboth, R. E. Ward, and G. Wilkening, TheEffect of Variations in the Refractive Index of Industrial Air uponthe Uncertainty of Precision Length Measurement, Metrologia30, 7-14 (1993).

[17] John S. Beers, Length Scale Measurement Procedures at theNational Bureau of Standards, Interagency Report IR 87-3625,National Bureau of Standards (U.S.) (1987).

[18] John S. Beers, A Gage Block Measurement Process Using SingleWavelength Interferometry, Monograph 152, National Bureau ofStandards (U.S.) (1975).

[19] Precision Gage Blocks for Length Measurement (Through 20 in.and 500 mm), ANSI/ASME B89.1.9M-1984, The American So-ciety of Mechanical Engineers, New York, NY (1984).

[20] J. S. Beers and C. D. Tucker, Intercomparison Procedures forGage Blocks Using Electromechanical Comparators, Intera-gency Report 76-979, National Bureau of Standards (U.S.)(1976).

[21] Charles P. Reeve, The Calibration of a Roundness Standard,Interagency Report 79-1758, National Bureau of Standards(U.S.) (1979).

[22] G. Schulz and J. Schwider, Interferometric Testing of SmoothSurfaces, Progress in Optics, Volume XIII, pp. 95-167, North-Holland Publishing Company (1976).

[23] Pulfrich, Interferenzmessaparat, Zeit. Instrument. 18, 261(1898).

[24] Charles P. Reeve, The Calibration of Indexing Tables by Subdivi-sion, Interagency Report 75-750, National Bureau of Standards(U.S.) (1975).

[25] Charles P. Reeve, The Calibration of Angle Blocks by Intercom-parison, Interagency Report 80-1967, National Bureau of Stan-dards (U.S.) (1980).

26] Standard Specification for Wire-Cloth Sieves for TestingPurposes, ASTM Designation E 11-87, American Society forTesting Materials, West Conshohocken, PA (1987).

675


About the authors: Ted Doiron and John Stoup aremembers of the Precision Engineering Division of theNIST Manufacturing Engineering Laboratory. TheNational Institute of Standards and Technology is anagency of the Technology Administration, U.S. Depart-ment of Commerce.

676

Volume102,Number6,November–December1997 …...Volume102,Number6,November–December1997...

Documents

Transcript of Volume102,Number6,November–December1997 …...Volume102,Number6,November–December1997...