VOTING SYSTEMS CERTIFICATION

In January 2017, the NCSBE published a document describing the Election Systems Certification Program.

A system must first meet federal standards (EAC approved for example) before it can be certified.

WHO SETS VOTING SYSTEM STANDARDS?

1. Federal Compliance Testing

The U.S. Election Assistance Commission (EAC) adopted Voluntary Voting System Guidelines (VVSG) in 2005 to recommend specifications against which voting systems can be tested

The EAC uses the VVSG to certify systems based on exhaustive tests of hardware and software attributes in respect to security, functionality, privacy, usability and accessibility.

2. North Carolina Certification Testing

Systems seeking certification in NC must pass testing to evaluate its functionality with respect to the specific practices of the State.

Systems are also evaluated for how they comply with legal requirements and administrative code, as well as integration with State Board regulations and policies.

VOTING SYSTEMS CERTIFICATION PROCEDURES

Receipt of Application

• Letter of Request for Certification • Receipt of Application• Functional Demonstration

Level I

• 4. Technical Data Package and Corporate Information• 5. Preliminary Review

Level II

• 6. Non-operating, Environmental Evaluation• 7. Simulated Election Event (S.E.E.)• 8. Public Demonstration Event• 9. Public Comment Period

Final Steps

• 10. State Board of Elections Final Recommendation• 11. Notification of State Board Decision

LEVEL I: APPLICATION AND EVALUATION

BASIC REQUIREMENTS FOR FUTURE VOTING SYSTEMS

Paper-based with precinct-level tabulator

ADA machines that produce a paper ballot and Ballot-on-Demand options that can feed into precinct scanners

Central, high-volume scanning product

Election reporting system and software that capable of producing flat file types to import directly into NC’s state reporting system

Documentation, training, and customer support for implementation and maintenance

REQUEST FOR CERTIFICATION (3.5.2 & 3.5.3)

Application submissions required the following information:

Vendors were invited to give a functional-demonstration at the NCSBE before continuing to the next stage

Equipment documentation Jurisdictional references

Technical design package Maintenance requirements

Testing documentation Ballot specifications

Corporate information Training/services available

LEVEL I: EVALUATION PROPOSAL (3.5.5 & 3.5.6)

After the functional demonstration, vendors were given permission to proceed with review of the system by an evaluation agent. The evaluation agent is an independent expert selected by the State Board of Elections to comply with GS 163A-1115.

GS § 163A-1115 (e) Prior to certifying a voting system, the State Board shall review, or designate an independent expertto review, all source code made available by the vendor pursuant to this section and certify only those voting systems compliant with State and federal law. At a minimum, the State Board's review shall include a review of security, application vulnerability, application code, wireless security, security policy and processes, security/privacy program management, technology infrastructure and security controls, security organization and governance, and operational effectiveness, as applicable to that voting system. Any portion of the report containing specific information related to any trade secret as designated pursuant to G.S. 132-1.2 shall be confidential and shall be accessed only under the rules adopted pursuant to subdivision (9) of subsection (f) of this section. The State Board may hear and discuss the report of any such review under G.S. 143-318.11(a)(1). (emphasis added)

NORTH CAROLINA’S EVALUATION AGENT

Section 231(b) of the Help America Vote Act (HAVA) of 2002 (42 U.S.C. §15371(b)) requires that the EAC provide for the accreditation and revocation of accreditation of independent, non-federal laboratories qualified to test voting systems to Federal standards.

Voting system vendors are responsible for ensuring that the system is tested—often through a federally accredited Voting Systems Test Laboratory or VSTL—to the required standards. Approval is issued at the state level, after which local jurisdictions may purchase the system.

LEVEL II: EVENTS AND DEMONSTRATIONSSIMULATED ELECTION EVENT (S.E.E.) AND PUBLIC DEMONSTRATION EVENT & COMMENT PERIOD

SIMULATED

ELECTION

EVENT

All election event workflow tasks necessary to conduct an election in North Carolina must be demonstrated and tested, including but not limited to:

The vendor was required to coordinate with agency staff designated by the State Board to conduct a simulated election event using the system to be certified. •An “end-to-end” test•Performed with ProV&V

setting up election

definitions

equipment and ballot

coding

ballot printing

generating equipment

media

ballot and equipment

testing

tabulation of election

results

importing election

results into NC’s

reporting system

canvassing and

adjudication procedures

recount procedures

final reporting by

precinct

SIMULATED ELECTION EVENT (SECTION 3.5.9)

Testing Process Notification of

Vendor Workflow Testing Protocol Ballot Preparation Ballot Review and Approval

Test Script Generation and

Test Deck Printing

Election Coding and Tabulation Software

Election Equipment Setup

Voting Demonstration

using the Test Deck

Vote Tabulation Review Ballot Preparation Election Results

Import and ReviewIntegration into

SEIMS

Adjudication and Canvassing

Recount Procedures

SIMULATED ELECTION EVENT (SECTION 3.5.9)

Equipment Needing Certification(Reviewed in Next Section)

Election Management System

Precinct Tabulators

Central Tabulators

ADA Accessible and Ballot Marking Devices

Ballot on Demand Solutions

Peripheral Hardware

Data Used In Tests

North Carolina elections

Wake and Cleveland Counties

Primary and General

Ballot markings by both hand and device

at least 84 ballot styles

Standardized quantities of ballots

PRO V&V TEST REPORT FINDINGS

The VSTL submitted that all test cases presented were passed and that Pro V&V considers the conditions of satisfaction to be met for the NC Certification Program by the following vendors:

7/12/17: ES&S -Voting System EVS5220

4/25/18: Clear Ballot

4/25/18: Hart InterCivic

PUBLIC DEMONSTRATION AND COMMENT PERIOD (SECTIONS 3.5.10 & 3.5.11)

The State Board’s voting systems certification team hosted all qualified vendors in a public demonstration event in Raleigh, NC in July 2018.

A public comment period was available from July 27 – August 10, 2018 to take online and mailed submissions. Few submissions named any vendor in particular in their

comments.

Most comments from the public focused on a preference for paper ballots, which is already a requirement met by systems in consideration.

Photo by Melissa Boughton, NC Policy Watch

VOTING SYSTEMS PENDING CERTIFICATIONSUMMARY AND REVIEW OF EQUIPMENT

CLEAR BALLOT

Headquarters in Massachusetts

System: ClearView1.4

EAC-certified 2/8/2018

Used in Washington and OregonSample Ballot, uses standard paper sizes

CLEAR BALLOT: CLEARVOTE 1.4

CLEAR BALLOT: CLEARVOTE 1.4

CLEAR BALLOT CLEARVIEW 1.4 – SYSTEM OVERVIEW

ES&S: EVS 5.2.2.0

Headquarters in Nebraska

System: EVS 5220

EAC-Certified 2/27/2017

Used in Michigan, Idaho, and UtahSample Ballot, uses unique paper sizes

ES&S: EVS 5.2.2.0

ES&S: EVS 5220 – SYSTEM OVERVIEW

HART: VERITY VOTING 2.2

Headquarters in Texas

System: Verity Voting 2.2

EAC-certified in 2/27/2016

Used in Minnesota and Mississippi

Sample Ballot, uses standard paper sizes

HART: VERITY VOTING 2.2

HART VERITY VOTING 2.2 – SYSTEM OVERVIEW

VOTING SYSTEM EQUIPMENT OVERVIEW

PRECINCT TABULATORS

Clear Ballot - ClearCast ES&S – DS200 Hart –Verity Scan

ADA-COMPLIANT MACHINES

Clear Ballot - ClearAccess ES&S - AutoMARK Hart –Verity Touch Writer

• Machines to be “ballot marking devices” – actual ballot will be scanned in precinct tabulator• Machines to allow for audio playback, ports for auxiliary assistive devices, tactile switches, low vision settings

BALLOT ON DEMAND

Clear Ballot - Clear Access ES&S – ExpressVote Hart InterCivic –Verity Print

• Marked ballot prints on blank piece of stock (8.5 inches wide)

• Attached printer

• Blank ballot with contests inserted (special size) and device itself marks and returns ballot.

• No separate printer

• Marked ballot prints on blank piece of stock (8.5 inches wide)

• Attached printer

CENTRAL SCANNERS

Clear Ballot - ScanStation ES&S – DS450/DS850 Hart InterCivic –Verity Central

Fujitsu fi-6800 Scanner Proprietary scanners

Mid and high volume options

Requires independent Dell desktop workstation

Cannon scanner

TABULATION AND REPORTING

Clear Ballot ES&S Hart InterCivic

Desktop only, standalone “kiosked” workstation

Browser-based system Unity’s Successor

CLEAR BALLOT CLEARVIEW 1.4 – COTS

Ergotron® Neo-Flex® Widescreen Lift Stand (33-329-085)

Computer stand required to use Dell OptiPlex all-in-one in portrait mode

HamiltonBuhl™ SchoolMate™ HA7 Headphones

Headphones

Inclusion Solutions Cardboard Voting Screen Privacy screen for accessible voting station

Storm Interface® EZ Access® Keypad (EZ08-222013)

Section 508-compliant keypad

Origin Instruments Breeze™ BZ2 sip-and- puff device with Headset (AC-0313-H2)

Sip-and-puff alternate entry device

APC™ Smart-UPS™ SMT1500 & SMT2200APC Smart-UPS

Uninterruptible power supply for printer(Recommended)

Pearstone™ USB 2.0 Type A Male to Type B Male Cable 6' (USB-AB6)

USB cable

Kingston® DataTraveler 100 G3 32 GB USB3.0 flash drive (DT100G3/32GB)

USB drive

SanDisk® Ultra Flair™ 32 GB USB 3.0 flash drive (SDCZ73-032G-A46)

USB drive

Fujitsu Central ScannerBrother, Okidata and HP PrintersDell OptiPlex 5250 & 7440; Dell PowerEdge T330; Dell Latitude 5580; Dell Precision T3620

Computer (options)

ES&S EVS 5220 – COTS

APC™ Smart-UPS™ SMT1500 Uninterruptible power supply (recommended)

AVID Headphones

Delkin USB drive

Delkin Compact Flash Memory Card

Proprietary System Central ScannerDell s2810dn; OKI B431D, B431DN PrintersDell OptiPlex 5250 & 7440; Dell PowerEdge T330; Dell Latitude 5580; Dell Precision T3620

Computer (options)

HART VERITY VOTING 2.2 – COTS

OKI B431/B432, C831dn, C911 Printer

Eaton 5P1500 UPS (recommended)

HP z230/2240 Computer

Cannon DR-G1100, DR-g1130 Central Scanner (options)

PENDING: FINAL RECOMMENDATION (SECTION 3.5.12)

Executive Director’s Final Recommendation

The appointed board may then consider and vote upon the certification at its discretion.

NEXT STEPS

NEXT STEPS

SBE FinalRecommendation

Notification to CBEs

County DemonstrationCounty Testing

CBE Recommendation to

BCCs County Commissioners

Approval

System Acquisition and Implementation

Receipt of Application

1.Letter of Request for Certification 2.Receipt of Application3.Functional Demonstration

Level I

4. Technical Data Package and Corporate Information5. Preliminary Review

Level II

6. Non-operating, Environmental Evaluation7. Simulated Election Event (S.E.E.)8. Public Demonstration Event9. Public Comment Period

Final Steps

10. State Board of Elections Final Recommendation

11. Notification of State Board Decision

SBE CBE

VOTING SYSTEMS CURRENTLY IN USE

DRE DECERTIFICATION

ELECTION DAY PRECINCTS

DRE DECERTIFICATION

ELECTION DAY ADA

DRE DECERTIFICATION

ONE-STOP

DRE DECERTIFICATION

ONE-STOP ADA

Thank You