114 A Method for Assigning Priorities to United States Measurement

1. Introduction

Nano-electrotechnologies are expected to be amongthe key technologies of the 21st century. They haveenormous potential for the development of new prod-ucts with exceptional performance. Nano-electrotech-nologies will enable society to take advantage of eco-nomic successes, as well as improvements in the quali-ty of life by using nano-enabled products. A strongmeasurements and standards infrastructure is essentialif the investments in nanotechnologies are to be suc-cessful for the delivery of useful products and services.[1,2] International commerce in nano-electrotechnolo-gies will require technically valid standards and relatedmeasurements that are suitable for use in any nation.

These standards must therefore be developed with inputfrom all stakeholders.

According to a recently published report ofSemiconductor Equipment and Materials International(SEMI) in cooperation with the SemiconductorIndustry Association (SIA) [3] and by the RNCOSGroup [4], the materials and equipment market fornanoelectronics was US$ 1.8 billion in 2005 and isexpected to be US$ 4.2 billion in 2010. The continuedrapid growth of this and other nano-electrotechnolo-gies-based industries has required increased interna-tional standardization activities to support equitableand efficient business models.

Volume 114, Number 4, July-August 2009Journal of Research of the National Institute of Standards and Technology

237

[J. Res. Natl. Inst. Stand. Technol. 114, 237-248 (2009)]

A Method for Assigning Priorities to UnitedStates Measurement System (USMS) Needs:

Nano-Electrotechnologies

Volume 114 Number 4 July-August 2009

Herbert S. Bennett

National Institute of Standardsand Technology,Gaithersburg, MD 20899

Howard Andres and JoanPellegrino

Energetics Incorporated,Columbia, MD 21046

[email protected]

In 2006, the National Institute ofStandards and Technology conducted anassessment of the U.S. measurement sys-tem (USMS), which encompasses all pri-vate and public organizations that develop,supply, use, or ensure the validity of meas-urement results. As part of that assess-ment, NIST collaborated with EnergeticsIncorporated to identify and authenticate723 measurement needs that are barriers totechnological innovations. A number ofthese measurement needs (64) are relevantto accelerating innovation and commer-cialization of nano-electrotechnologies. Inthis paper, we apply the taxonomy from a2008 international survey that establisheda global consensus of priorities for stan-dards and measurements in nano-elec-trotechnologies to rank in priority orderthe relevant 64 USMS-identified measure-

ment needs. This paper presents a methodfor assigning priorities that is statisticallybased and represents a global consensus ofstakeholders. Such a method is neededbecause limited resources exist to addressthe large number of measurement needs innano-electrotechnologies, and the mostcritical measurement needs should beaddressed first.

Key words: Borda count method; meas-urement needs; median method; nano-elec-trotechnologies; priorities; rankings; stan-dards; statistical significance; UnitedStates Measurement System.

Accepted: May 12, 2009

Available online: http://www.nist.gov/jres

Applications of nano-electrotechnologies include: [5]• Analytical equipment and techniques for measure-

ment of electrotechnical properties• Fabrication tools for integrated circuits (electronic,

photonics, and optoelectronic)• Nano-structured sensors• Nano-electronics, materials and devices• Optoelectronics• Optical materials and devices• Organic (opto) electronics• Magnetic materials and devices• Radio frequency devices, components, and systems• Electrodes with nano-structured surfaces• Electrotechnical properties of nanotubes/nanowires• Fuel cells• Energy storage devices (e.g., batteries)• Bioelectronic applications• Nano-enabled solar cells

Due to the large number of potential applications fornano-electrotechnologies and to the limited resourcesfor development of standards, there is a need to priori-tize future standardization work and make certain thatthe most important standards are developed first. Arecent international effort in this regard is the NIST-Energetics-International Electrotechnical Commission(IEC) Technical Committee (TC) 113 InternationalSurvey. [6] The analysis described in this paper buildson this previous effort to prioritize standards and theirassociated measurement needs (MNs) by applying amethod for assigning priorities that is statistically basedand represents a global consensus of stakeholders.

1.1 Objectives

The objective of this paper is to use the results fromprevious efforts and analyses to demonstrate as a proof-of-concept a method for assigning priorities to severalaction items, which in this paper are USMS MNs.Section 2 summarizes the background, origin, struc-ture, methodology, demographics, and results of priorefforts to prioritize measurement needs, including a2006 assessment of the U.S. measurement system [7]and the IEC international Survey. [6] Section 3 containsthe procedures by which we assign priorities to the 64USMS MNs on nano-electrotechnologies identified inthe 2006 assessment based on the Survey taxonomy.Section 4 contains a summary of the major results.Finally, Appendix A contains a listing of the 64 casestudies of USMS MNs on nano-electrotechnologies.

Definitions:Nanotechnology is the understanding and control of

matter at dimensions between approximately 1 and 100nanometers, where unique phenomena enable novelapplications. Encompassing nanoscale science, engi-neering, and technology, nanotechnology involvesimaging, measuring, modeling, and manipulating mat-ter at this length scale. Dimensions between approxi-mately 1 and 100 nanometers are known as thenanoscale. Unusual physical, chemical, and biologicalproperties can emerge in materials at the nanoscale.These properties may differ in important ways from theproperties of bulk materials and single atoms or mole-cules. [8]

Nano-electrotechnologies include the followingareas at the nanoscale: nanostructured sensors; nano-electronics, nano-materials and nano-devices; optoelec-tronics; optical materials and devices; organic (opto)-electronics; magnetic materials and devices; radio fre-quency devices, components and systems; electrodeswith nanostructured surfaces; electrotechnical proper-ties of nanotubes/nanowires; analytical equipment andtechniques for measurement of electrotechnical proper-ties; patterning equipment and techniques; masks andlithography; performance, durability, and reliabilityassessment for nanoelectronics; batteries; fuel cells;and bioelectronic applications. [9]

2. Nanotechnology Measurement andStandards: Assessment andPrioritization

2.1 2006 Assessment of the U.S. MeasurementSystem

As the national measurement institute for the UnitedStates, the National Institute of Standards andTechnology (NIST) assists all stakeholders in selectedfields with measurement and standards needs. The goalof NIST’s involvement is to enhance efficiency andproductivity and to increase the rate of technologicalinnovation. NIST is not a regulatory agency, but ratherserves as a neutral third party often providing technicalinput in matters related to measurements and standardsto a variety of customers such as standards committees,regulatory agencies, other government agencies, uni-versities, and the private sector where appropriate. In2006, NIST accepted the challenge to evaluate whetherthe U.S. measurement system (USMS) is meeting thenation’s measurement needs and produced an assess-ment detailing the findings. [7] Nanotechnology was


238

one of the sector areas assessed in terms of measure-ment needs for accelerating technological innovationand commercialization.

As part of that assessment, NIST collaborated withEnergetics Incorporated to identify and authenticate723 measurement needs that are barriers to technologi-cal innovations. These measurement needs werederived from case studies and a review of technologyroadmaps and reports in the public literature. AppendixB of the June 2006 USMS Assessment Report [7] con-tains 342 case studies of measurement needs (MNs), ofwhich 64 cases studies concerned nano-electrotech-nologies.

Based on their evaluation of measurement needs, theauthors of the June 2006 USMS Assessment Report [7]made the following observations about nanotechnologies:• Nanotechnology is unique among the sector/tech-

nology areas in its high demand for new advancedmeasurement instrumentation, which is needed toachieve accurate, high-resolution characterizationof physical, chemical, and biological properties ofmaterials at nanometer dimensions.

• Industry is limited not only in its ability to measurekey parameters but also in its ability to identifywhich key parameters must be measured to meetanticipated regulations.

• The absence of measurement tools with the capabil-ity to measure properties of nanomaterials and nan-odevices as they relate to functional performanceand to make such measurements at speed are imped-iments to realization of nanotechnology products.

• Public-sector measurement providers that are linkedto applied research sectors, production communi-ties, market drivers, and user issues can help accel-erate the innovative process.

• Innovative approaches to the measurement ofnanoscale physical and chemical properties are keyto technological innovation for nanotechnology,especially where the fundamental limits of currentmeasurement techniques are being approached.

Industry technology roadmaps are an importantsource of measurement needs pertinent to nanotech-nologies. These include the 2007 InternationalTechnology Roadmap for Semiconductors [1] and the2003 Chemical Industry Roadmap for Nanomaterialsby Design. [2] Prepared in conjunction with the USMS2006 Assessment, the USMS Technology RoadmapReview Summary Report identified a number of meas-urement needs relevant to nanotechnologies, some ofwhich are shown. [7]

• Sensing and detection devices operating at thenanoscale are likely to have myriad applications,including: detection of chemical, biological, radio-logical, and explosive elements; detection and treat-ment of infection, nutrient deficiency, and otherhealth problems; tracking of food pathogens andagricultural products; nanoseparation andnanobioreactors; ensuring food safety; environmen-tal monitoring; advanced protective clothing and fil-ters and remediation of attacks; and creating anti-fouling nanosurfaces (e.g., packaging).

• A broad range of measurement needs associatedwith the understanding, characterizing, synthesiz-ing, and manufacturing of new nanomaterials: char-acterization tools, methods, and instruments forproperties measurement; tool development infra-structure; reference standards and protocols for syn-thesis and analysis protocols; robust measurementtools for manufacturing; characterization, measure-ment, and simulation probes for use during synthe-sis; and measurements for environmental, health,and safety impacts of nanomaterials.

• Measurement capabilities to further developmentand application of nanostructures as energy carriers(optimized energy transport); characterizationmethods, theory, imaging tools, and simulation andmodeling to link nanoscale structure and functionfor nanomaterial assembly and architecture with thedesign of new materials for energy applications; andmeasurements to aid evaluation and development ofcarbon nanotubes for hydrogen storage.

2.2 NIST-Energetics-IEC TC 113 InternationalSurvey

In 2008, the IEC TC 113 on nano-electrotechnolo-gies invited members of the international nanotech-nologies community to respond to a Survey to identifythose nano-electrotechnologies relevant to electronicsand electrical products and systems for which standardsare critically needed to accelerate innovation. Theresulting Survey paper [6] contains the analyses of 459Survey responses from 45 countries.

The Survey represents one way to begin building aconsensus on a framework leading to nano-electrotech-nologies standards development by standards organiza-tions and national measurement institutes. The expecta-tion was that responses to the Survey would enable theIEC TC 113 to:• set procedures for ranking proposals and associated

documents for new work in priority order;


239

• identify members for work groups on standards andassociated documents; and

• make informed responses to proposals from IECNational Committees.

The distributions of priority rankings from all 459respondents are such that there are perceived distinc-tions with statistical confidence between the relativeinternational priorities for the several items ranked ineach of the following five Survey category types:

1) Nano-Electrotechnology Properties, 2) Nano-Electrotechnology Taxonomy: Products, 3) Nano-Electrotechnology Taxonomy: Cross-Cutting

Technologies,4) IEC General Discipline Areas, and5) Stages of the Economic Model.

Table 1 illustrates the category types and taxonomyemployed in the Survey. We use ni in the next paragraphto denote the number of items listed in each categorytype. For example, ni equals 6 for category typeProperties. One of the primary goals was to determinea consensus prioritization among the items listed foreach of the category types. With this goal in mind, theSurvey required the respondents to rank all items foreach of the five category types, with no ties allowed.

Tables 2 through 4 from the Survey paper [6] showthe consensus priorities for each of the first three cate-gory types as determined by a traditionally weightedscoring technique called the Borda count. [10] TheSurvey paper provides complete details of the methodand results of the statistical analyses. Applying this pro-cedure to the Survey category types, the followingBorda score-weights were assigned: the first-placeditems (highest priority or most significant) on everyballot receive scores of ni , the second-placed itemsreceive scores of ni – 1, and so forth, until the lowestpriority or least significant items on the ballot receivescores of 1. Scores were assigned to each of the 459ballots from the respondents individually and thensummed over all ballots within the category type ofinterest.

Items were ranked in descending order by the Bordascore; i.e., the highest score is the “winner.” In short,the Borda score is a weighted mean with a particularassignment of weights to ballot positions. These Bordacount orderings are referred to throughout the paper asthe “global consensus” orderings. The global consensusorder may not be the same as the order when only rank1 votes are considered. For example, Fabrication Toolsin Table 4 received 109 rank 1 votes, 61 rank 2 votes,…, and 44 rank 8 votes. All of the remaining 7 items in

Table 4 received fewer than 109 rank 1 votes. Themedian rank of the underlying random variable wasestimated to be 3 ±0.29. The global consensus is thatFabrication Tools is second to Sensors as a priorityactivity. Appendix B of the Survey paper [6] containsthe definition for the 95 % confidence interval (CI) anddescribes the methodology in detail.

3. Assigning Priorities to USMSMeasurement Needs for Nano-Electrotechnologies

As noted earlier, there is a large number of measure-ment needs for nano-electrotechnologies and limitedresources to address them (both domestically and glob-ally). Consequently, there is a need to rank these in pri-ority order so that resources can be applied to address-ing those needs that are most important to acceleratinginnovation.

For this analysis we assumed that the set of 64 MNsidentified in the 2006 USMS assessment may serve asa proxy for the universe of nano-electrotechnologymeasurement needs. We then apply a process to analyzeand rank this set in priority order using the taxonomydeveloped for the 2008 NIST-Energetics-IEC TC 113Survey on Nano-Electrotechnologies. [6] As a proof-of-concept, we use the following method for assigningpriorities to the 64 nano-electrotechnology USMS casestudies of MNs listed in Appendix A.

3.1 Tagging Methodology for USMSMeasurement Needs

A “tagging” process was employed to provide a con-sistent set of information from each MN. The tags cor-respond to the first three category types and rankeditems given in Table 1, namely Properties, Products,and Cross-Cutting Technologies. Through this processwe were able to uniformly gather a set of priority infor-mation for each MN that corresponded to the sameranking choices given to the Survey respondents.

A set of “items-tags” (i.e., ranked items from theProperties, Products, and Cross-Cutting Technologiescategory types) was selected for each of the 64 USMSMN case studies. Selection of tags was based on bestscientific and engineering judgment, given the informa-tion presented in the case studies. Only one ranked itemwas assigned from each Survey category type to eachUSMS MN. Examples of the actual tags assigned tocase studies for this analysis are shown in Table 5. For


240


241

Table 1. Category Types and Rank Items Employed in the Survey [denotes abbreviation in data tables]

1. Properties• Electronic and Electrical [Electronic]• Optical [Optical]• Biological [Biological]• Chemical [Chemical]• Radio Frequency [Radio]• Magnetic [Magnetic]

2. Products• Energy (production, conversion, and storage) [Energy]• Medical Products [Medical]• Computers (PDA and similar, laptop, desktop, mainframe) and Computer Peripherals (printers, monitors/displays,

etc.) [Computers]• Telecommunication and Data Communications (wireless and wired-physical connection) [Telecom]• Security and Emergency Response Devices and Applications [Security]• Multimedia Consumer Electronics [Multimedia]• Household and Consumer Applications [Household]• Transportation (sea/water, ground, air, space) [Transportation]

3. Cross-Cutting Technologies• Sensors (chemical, physical, mechanical, etc.) [Sensors]• Fabrication tools for integrated circuits (electronic, photonic, optoelectronic, and mechanical) [Fab. Tools]• Nano-Electromechanical systems [NEMS]• Performance and reliability assessment for nanoelectronics [Performance]• Analytical equipment and techniques for measurements of electro-technical properties [Analytic Eq.]• Environment, Health, and Safety (EHS) applications and effects [EHS]• Instrumentation (test equipment and industrial process control for use in fabrication) [Instrumentation]• Optical technologies (optoelectronics and illumination) [Optical Tech.]

4. General Discipline Areas• Measurement and Performance [Measurement]• Design and Development [Design]• Health, Safety, and Environment (HSE) [HSE]• Dependability and Reliability [Dependability]• Electromagnetic Compatibility [Compatibility]• Terminology, Nomenclature, and Symbols [Terminology]

5. Stages of Economic Model• Basic Technical Research [Research] • Technology Development (prototype development) [Development] • Initial deployment [Deployment] • Commercialization (large-scale, high-volume manufacturing) [Commercialization]• End of initial use by the Customers-Consumers (End of Initial Usefulness) [End-of-Usefulness]• End-of-Life (disposing and recycling) [End-of-Life]


242

Table 2. Consensus Priority Rankings for Properties [6]

Raw Data Borda GlobalMedian and Borda Consensus

Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6 95 % Cl Score Rank

Electronic 292 57 58 26 13 13 1(+0.07) 2,386 1Optical 17 115 112 105 78 32 3(±0.15) 1,628 2Biological 68 73 68 75 77 98 4(±0.22) 1,522 3Chemical 37 86 70 68 113 85 4(±0.22) 1,447 4Radio 34 83 69 78 63 132 4(±0.29) 1,387 5Magnetic 11 45 82 107 115 99 4(±0.15) 1,269 6

Table 3. Consensus Priority Rankings for Products [6]


Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6 Rank 7 Rank 8 95 % Cl Score Rank

Energy 130 94 69 52 34 37 18 25 3(±0.22) 2,680 1Medical 85 103 85 57 41 45 26 17 3(±0.22) 2,564 2Computers 109 63 60 59 57 52 31 28 3(±0.22) 2,442 3Telecom 57 82 72 89 72 43 29 15 4(±0.22) 2,397 4Security 25 43 62 67 75 77 51 59 5(±0.22) 1,900 5Multimedia 22 39 47 59 72 65 83 72 5(±0.22) 1,747 6Household 20 12 39 30 47 76 119 116 7(±0.22) 1,398 7Transportation 11 23 25 46 61 64 102 127 6(±0.22) 1,396 8

Table 4. Consensus Priority Rankings for Cross-Cutting Technologies [6]


Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6 Rank 7 Rank 8 95 % Cl Score Rank

Sensors 100 94 60 49 51 45 34 26 3(±0.22) 2,496 1Fab. Tools 109 61 66 52 47 40 40 44 3(±0.29) 2,387 2NEMS 59 71 59 58 65 45 46 56 4(±0.29) 2,156 3Performance 55 54 58 57 57 61 60 57 5(±0.29) 2,039 4Analytical Eq. 30 57 54 70 80 74 58 36 5(±0.22) 2,007 5EHS 71 40 45 39 48 54 66 96 5(±0.29) 1,895 6Instrumentation 13 39 58 73 60 71 84 61 5(±0.22) 1,772 7Optical Tech. 22 43 59 61 51 69 71 83 5(±0.29) 1,772 7

this proof-of-concept study, only three respondents par-ticipated in the tagging process.

3.2 Tagging Results

We developed a set of histograms to illustrate thedistributions of Properties, Products, and Cross-CuttingTechnologies ranked “items-tags” that the 3 respon-dents assigned to each of the 64 USMS MN case stud-ies. These are shown in Figs. 1 through 3. These his-tograms give rise to some general observations and alsoprovide a basis for comparison with the NIST-Energetics-IEC TC 113 Survey, as outlined below.• Properties (Fig. 1) – The properties most frequent-

ly identified in the USMS MN cases studies fall withinthe areas of Optical and Electronic. This is somewhatcomparable to the priorities identified in the Survey(refer to Figs. 4 and 9 in [6]). However, the overwhelm-ing priority from the Survey was Electronic properties,compared with the USMS MN case studies, for whichElectronic and Optical properties share large portionsin the distribution of tags for properties. In the Survey,Optical properties received low priority rankings.Biological properties have a moderate but similar sharein both the USMS MN tagging and the Survey.• Products (Fig. 2) – The Product category most fre-

quently identified in the USMS MN case studies wasComputers, followed (but not closely) by Medicalproducts. However, the product categories ofComputers, Medical, and Energy in the Survey itself(refer to Figs. 5 and 10 in [6]) all have relatively highpriority rankings. A possible reason for this is the morerecent emphasis worldwide on energy as a prioritycompared to when the 2006 USMS report was conduct-ed. While important, energy was not viewed to be atsuch a critical juncture in 2006 as it is today. In themedical field, recent rapid advances in innovativefields may be fueling a greater need for standards.

Telecommunications received a relatively large share ofhigh priority rankings, compared with the USMS MNcase studies, where it only appeared in a few cases.• Cross-Cutting Technologies (Fig. 3) – In this cate-

gory, Instrumentation has the highest number of assign-ments, followed by Fabrication Tools and AnalyticalEquipment. When compared with the Survey (refer toFigs. 6 and 11 in [6]), only Fabrication Tools followsthe same pattern. In the Survey, Instrumentation actual-ly was ranked at the lowest priority. Sensors was high-est in priority among the eight items for Cross-CuttingTechnologies in the Survey, but very seldom assignedas a tag for the USMS MN case studies. In somerespects, this may be an artifact of attempting to matchthe Survey categories to the USMS MN case studies;the latter of which were developed with a different per-spective and not for the taxonomy used in the Survey.This is particularly true where overlap may existbetween Instrumentation for testing and controllingfabrication processes or Analytical Equipment formeasuring electro-technical properties. It may be diffi-cult to distinguish which category is more appropriatein some cases. This sort of anomaly might be removedif the case studies were written more clearly or withthese particular categories in mind. Another anomaly isCross-Cutting Technology Item Environmental Healthand Safety (EHS), which was identified as a relativelyhigh priority in the Survey, but was identified in only asmall number of the USMS MN case studies. This isanother case of a change in global priorities and inter-ests—with the advent of more nanotechnologies intothe marketplace and on the drawing board, interest intheir safety and impact on the environment hasincreased.

We also compared the distributions of assignedUSMS MN tags for Properties, Products, and Cross-Cutting Technologies with the high priority rankings


243

Table 5. Selected Examples of Tagging for USMS MN Case Studies

USMS MN Case Study Title Property Tag Product Tag Cross-cutting Technology Tag

Nanomagnetic MRI Contrast Optical Medical EHS Application and EffectsAgents (p. A-52)

Cell-Based Analysis Using Electronic and Electrical Medical Fabrication ToolsLab-on-a-Chip Technologies(p. A-12)

Nanoscale Chemical Optical Computers Fabrication ToolsCharacterization of AdvancedMaterials (p. A-63)


244

Fig. 1. Tag Distribution for Properties.

Fig. 2. Tag Distribution for Products.


245

Fig. 3. Tag Distribution for Cross-Cutting Technologies.

Fig. 4. Distribution of Total Scores for the 2006 USMS MNs in Nano-electrotechnologies.


246

Table 6. Priorities for USMS Nano-Electrotechnology Measurement Needs by Total Score. Based on Borda Global Consensus Rank. The lowerthe total score the higher the priority.

TOTAL SCOREAppendix A: Based on Borda GlobalPage No. Measurement Need (MN) Title Consensus Rank

A-1 Carbon Nanotube Materials 15A-2 Quantum Computing 18A-3 Nanoscale Integrated Circuits: Dimensional Control 18A-4 Advanced CMOS Gate Stacks for Next Generation Integrated Circuit Devices 18A-5 Top Down Micro/Nano Manufacturing 20A-6 Nanostructured Materials for Photovoltaic 21A-7 High Accuracy Dimensional Metrology for Manufacturing 21A-8 Integrated Circuit Overlay Metrology 21A-9 Sub-50 nm Lithography 21A-10 Small Particle Monitoring For Advanced Semiconductor Manufacturing 21A-11 Advanced DNA Analyses Using Lab-on-a-Chip Technology 22A-12 Cell-Based Analysis Using Lab-on-a-Chip Technologies 22A-13 Sub-10 nm SEM Metrology Tools 23A-14 Nanoscale Biological Imaging 24A-15 Compound Semiconductor Cluster Tools 25A-16 Molecule-Based Nanoelectronics 26A-17 Nanomanufactured Components 26A-18 Multi-Layer Nanostructures for Electronic and Photonic Devices 26A-19 In-line Inspection and Factory Control Equipment 26A-20 Interfacial Characterization Instrumentation 26A-21 Semiconductor Industry Defect Metrology Tools 26A-22 Nanoimprint Lithography (NIL) 27A-23 Nanocrystal Biophotonic Sensors 27A-24 Current Flow in Nanoscale Electronic Devices 27A-25 Next Generation Electrical Instrumentation 29A-26 Next-Generation Active Nanodevices 30A-27 Single Molecule Optical Measurement 30A-28 Integrated Circuit Optical Linewidth Metrology 30A-29 Single Biomolecule Detection, Classification, and Measurement 30A-30 Integrated Circuit Photomask Metrology 31A-31 Nanomanufacturing 32A-32 Sidewall Characterization Instrumentation 32A-33 Health Care/Nanotechnology - Cancer Diagnosis and Treatment 33A-34 Advanced Force Measurements in Nanotechnology 33A-35 Advancing the Fundamental Science of Nanobiotechnological Systems 33A-36 In-line/Real-time Analytic Tools for Measuring Sub-10 nm Defects 34A-37 Dopant Distribution Instrumentation 34A-38 Atomic Mapping Instrumentation 34A-39 MEMS 34A-40 Dimensionally Critical Nanomanufacturing 35A-41 Airborne Contamination in Semiconductor Wafer Processing 36A-42 Carbon Nanotube Identification 36A-43 Multilayer Film Structures for Electronics and Optics Industries 36A-44 Novel Materials for Nanoscale Diffusion Barriers in Microelectronics 36A-45 Next-Generation Optical Microscopes 36A-46 Nano-Scale Drug Delivery 38A-47 Directed Nanoscale Assembly 39A-48 Strained-Layer Engineering for High Performance Electronic and Optoelectronic Devices 39A-49 Instrumentation for Measurement of Electrical Properties at the Nanoscale 39A-50 SEM and AFM Modeling for Semiconductor Electronics and Nanotechnology 40A-51 Micro/Nano-Technology 40A-52 Nanomagnetic MRI Contrast Agents 42A-53 Atomic-Precision Imaging to Aid Development of New Materials 42A-54 Hard Disk Stack Metrology 43

from the Survey. Table 6 illustrates the results of thisanalysis.

The scorings in the column on the right in Table 6show the total score for each MN based on the SurveyBorda global consensus ranks. This total score is thesum of the Borda ranks for each of the three rankedSurvey items which the respondents assigned to anMN. That is, each MN has three tags, one each fromProperties, Products, and Cross-cutting Technologies.The minimum total score that the three respondentscould assign a given MN is 3(1+1+1) = 9 (highest pri-ority). The maximum total score that they could assignis 3(6+8+8) = 66 (lowest priority). Thus, a low score inthe far right column indicates that this USMS MN hasa very high priority based on the Survey’s global con-sensus. Alternatively, a high score indicates that thisUSMS MN has a very low priority based on theSurvey’s global consensus. For example, the lowestscore of 15 received for the MN “Carbon NanotubeMaterials” indicates that the tagging for Properties,Products, and Cross-Cutting Technologies was highlycorrelated to high rankings in the original Survey. Thehigh score of 54 received for MN “Self Assembly ofSoft Nanomaterials” indicates that tagging results forthis MN were the least correlated with high-rankeditems in the original Survey.

The standard deviation for the total scores given inTable 6 is ±9. Figure 4 shows the distributions for thenumber of MNs receiving totals scores in the bandswith total score widths of 10. The first bar is the num-ber of MNs with scores between 9 and 19; the secondbar is the number of MNs with scores between 20 and30; and so on, with the fifth bar having scores between53 and 66. Figure 4 suggests that, using the taggingprocess, the Survey respondents would assign 4 MNs

the highest priority and 1 MN the lowest priority. Theywould assign 25 MNs high priority and 12 MNs lowpriority. The middle band has 22 MNs, indicating thatthose MNs are neither high nor low priority. Becausethe MNs are distributed among all 5 bands, we con-clude that the set of 64 MNs correlates with the priori-ties of the Survey respondents.

4. Conclusions

We have successfully demonstrated as a viableproof-of-concept a method for placing in priority orderUSMS MNs by assigning ranked items from an inde-pendent survey to the USMS MNs. Our analyses sug-gest that from the perspectives of the 459 Surveyrespondents the priority ranking of the 64 USMS MNcase studies for nano-electrotechnologies given inAppendix A is consistent with the Survey’s global con-sensus rankings. That is, by comparing the taggingselections with those of the Borda global consensusrankings in the Survey, we have established that theranked set of 64 MNs correlates with the priorities ofthe Survey respondents.

Three additional considerations are in order. First,the case studies used in the 2006 USMS assessmentwere not written with the Survey tagging concept inmind. As a result, interpretation of the case studies toidentify the most appropriate tags may be in some casesrelatively subjective. Second, as might be expected,global priorities for business, energy, medicine, envi-ronment, and other areas have changed since 2006.This change in global priorities may be reflected in afew cases by differences between the ranked items inSurvey taxonomy categories and the content of some


247

Table 6. Priorities for USMS Nano-Electrotechnology Measurement Needs by Total Score. Based on Borda Global Consensus Rank. The lowerthe total score the higher the priority. (Continued)

TOTAL SCOREAppendix A: Based on Borda GlobalPage No. Measurement Need (MN) Title Consensus Rank

A-55 Carbon Nanotubes (CNTs) 44A-56 Toxicology of Nano-Particles in Biological Systems 44A-57 Next Generation Electrotechnical Products, Components, and Raw Materials 45A-58 Magnetic Data Storage 45A-59 Nanomanufactured Components in Complex Fluids 46A-60 Spin Metrology Tools 47A-61 Hard Disk Sheet Magnetoresistive 48A-62 Scanning Electron Microscope Nanocharacterization 49A-63 Nanoscale Chemical Characterization of Advanced Materials 51A-64 Self Assembly of Soft Nanomaterials 54

USMS MN case studies. Third, the USMS MN casestudies were written by a narrow segment of the U.S.measurement community, whereas the Survey wasdeveloped and responded to by a much broader interna-tional measurement community. Even with these con-siderations, the process was still able to provide aviable proof-of-concept.

5. Website For Downloading the 64 USMSMNs on Nano-electrotechnologies

The one-page summaries of the 64 nano-electrotech-nology case studies are a subset of the measurementneeds given in Appendix B of the June 2006 USMSAssessment Report [7]. The Website for downloadingthe 64 nano-electrotechnology case studies ishttp://nvl.nist.gov/pub/nistpubs/jres/114/4/Appendix-A-for-NIST-Energetics-USMS-MN-Priorit ies-06Jul09.pdf

Acknowledgments

The authors thank William Anderson, StephenKnight, Joaquin Martinez, and Nicholas Paulter of theElectronics and Electrical Engineering Laboratory atthe National Institute of Standards and Technology(NIST) for providing the funds to support the interna-tional survey on which the results of this paper dependand Clare Allocca, Director of the NIST USMS Office,for her many contributions to the USMS process and itsongoing assessment.

6. References

[1] Semiconductor Industry Association, The InternationalTechnology Roadmap for Semiconductors, 2007 edition,Executive Summary. SEMATECH:Austin, TX, http://www.itrs.net/Links/2007ITRS/Home2007.htm (January 2009).

[2] 2003 Chemical Industry Roadmap for Nanomaterials byDesign: From Fundamentals to Function, http://www.chemicalvision2020.org/nanomaterialsroadmap.html (March 2009).

[3] SEMI-SIA Report Publication Date: November 2005,http://www.semi.org/en/Press/P036750 (January 2009).

[4] RNCOS Group Research Report # RNC1125; Publication Date:February 2007, http://www.electronics.ca/reports/nanotechnology/world_market.html (January 2009).

[5] TC 113, Nanotechnology standardization for electrical and elec-tronic products and systems, is born, http://www.iec.ch/online_news/etech/arch_2007/etech_0507/spotlight.htm?mlref=etech (July 2009).

[6] H. S. Bennett, H. Andres, J. Pellegrino, W. Kwok, N. Fabricius,and J. T. Chapin, Priorities for Standards and Measurements toAccelerate Innovations in Nano-electrotechnologies: Analysisof the NIST-Energetics-IEC TC 113 Survey, accepted for pub-

lication in the NIST Journal of Research, Volume 114, Issue 2,

March-April 2009. On-line at http://www.nist.gov/eeel/semiconductor/upload/NIST_Energetics_Survey.pdf (March 2009).

[7] An Assessment of the United States Measurement System:Addressing Measurement Barriers to Accelerate Innovation(NIST Special Publication 1048), June 2006, available fordownload at http://usms.nist.gov/usms07/index.html.

[8] What is Nanotechnology?, http://www.nano.gov/html/facts/whatIsNano.html (January 2009).

[9] IEC TC 113 Strategic Policy Statement, http://www.iec.ch/cgi-bin/getsps.pl/113.pdf?file=113.pdf (January 2009).

[10] K. J. Arrow, Social Choice and Individual Values, YaleUniversity Press, New Haven, 2nd ed. (1963).

About the authors: Herbert S. Bennett is a NISTFellow and Executive Advisor in the SemiconductorElectronics Division of the NIST Electronics andElectrical Engineering Laboratory and serves as amember of the IEC TC 113 Chairman’s AdvisoryGroup. Howard Andres is an Engineering ResearchAnalyst at Energetics Incorporated. Joan Pellegrino isa Program Director and Senior Consultant atEnergetics Incorporated. The National Institute ofStandards and Technology is an agency of the U.S.Department of Commerce.


248

114 A Method for Assigning Priorities to United States Measurement

Documents

Transcript of 114 A Method for Assigning Priorities to United States Measurement