Oregon Central Voter Registration Project

OCVR Eval Process V3 6-25-03 OCVR-BC

Oregon Central Voter Registration Project Evaluation Process for the Request for Proposals

June 25, 2003

Prepared by the Office of the Secretary of State for Oregon This document outlines the internal evaluation process and procedures for members of the Evaluation Team tasked with assessing Request for Proposal (RFP) submissions for the Oregon Central Voter Registration Project (OCVR).


The RFP documents the objectives and requirements for this project, outlines the general conditions and specific standards and details the format to be used by proposers in responding to the RFP. This Evaluation Framework does not repeat the RFP material and accordingly, all evaluators should familiarize themselves with the RFP. Capitalized terms in this document have the same meaning as defined in the RFP.

This document is organized into five sections:

Background – provides an overview to date of the process Selection Process – describes the stages and components of the evaluation Evaluation Teams – describes the roles and responsibilities of the Evaluation Teams Schedule of Events – outlines the expected timelines and activities of the evaluation process List of Team Members 1. Background The Secretary of State is one of three Constitutional offices established at statehood. The Secretary is the auditor of public accounts, registrar of corporate records, archivist of the state’s records and Oregon’s chief elections officer. The Secretary of State is comprised of the Executive Office, Audits, Archives, Business Services, Corporation, Elections, Information Systems, Cultural Trust and Personnel Resources Divisions. Each of these divisions provides diverse services to state agencies, other public entities and the citizens of the State of Oregon. The Secretary of State is Oregon’s Chief State Election Officer. Under state legislation passed in 2001 (HB 2002) the Secretary was directed to establish a centralized statewide voter registration system, and federal legislation passed in 2002 [PL 107-252] requires each state to “implement a single, uniform, official, centralized, interactive and computerized statewide voter registration list defined, maintained and administered at the state level.” [PL 107-252, Section 303 (a) (1) (b)] The Secretary of State, Information Services Division (ISD) sought proposals from qualified Contractors to acquire and implement an integrated voter registration and election management system for thirty-six (36) Oregon counties and the Secretary of State Elections Division (Elections). The Secretary of State will consider proposals for all types of systems that meet the requirements of this RFP (i.e. Application Service Provider (ASP), Commercial off the Shelf System (COTS), Custom Developed, or any combination). The final due date for all proposed deliverables for all phases are July 1, 2006. Each proposer is required to submit a proposal describing in detail the technical and cost aspects of the plan by which it proposes to meet the requirements of the RFP. Proposal components include:


• Technical proposal • System security proposal • Cost proposal

The due date for submitting proposals to the Secretary of State’s Purchasing Office is 2 p.m. on June 26, 2003. The schedule as set out in the RFP calls for selection of the Successful Proposer by August 1, 2003 with contract execution on or before August 31, 2003. 2. Evaluation and Scoring Process The proposal submission and evaluation process is outlined in Sections 12-14 of the RFP. As outlined in the RFP, each proposer is to submit its approach and plans relating to the elements of the project that will be evaluated. These elements are identified in the RFP in Section 14 as follows:

Evaluation Criteria

Points to be Allocated

Initial responsiveness to mandatory requirements (Sections 12 and 13) Pass/Fail High level and functional requirements (Sections 7 and 8) 275 Mandatory proposal requirements (Section 13) 300 System security proposal (part of Sections 7 and 8) 75 Oral Presentations (Section 12.9) 100 Demonstrations (Section 12.9) 100 Cost Proposals (Section 15.3.5) 150

The evaluation process will consist of five steps: Step 1: Mandatory requirements where proposals will be evaluated for completeness and compliance. (Pass/Fail) Step 2: Technical and business requirements where proposals will be evaluated for completeness, compliance, process, experience, customer references, and technical merit. (650 Points Possible for Sections 7, 8, and 13) Step 3: Oral presentations will be evaluated for completeness, compliance, company, experience, and technical merit. (100 Points Possible for Section 12.9, separate from demonstrations)


Step 4: Product demonstrations will be evaluated for completeness, compliance, company, experience, and technical merit. (100 Points Possible for Section 12.9, separate from presentations) Step 5: Financial requirements where proposals will be evaluated for completeness, compliance, economic and financial merit. (150 Points Possible in Section 15) All proposals will compete against each other and will be evaluated separately in arriving at the overall score of one thousand (1,000) points for technical, economic, and financial merit. 2.1 Scoring Criteria

Points Allocated

Rating Criteria

5 points Excellent An excellent response will be a highly comprehensive, excellent reply that meets all of the requirements of the areas within that category. In addition, the response covers areas not originally addressed within the RFP category and includes additional information and recommendations that would prove both valuable and beneficial to the agency. This response is considered to be an excellent standard, demonstrating the proposer’s authoritative knowledge and understanding of the project.

4 points Very good A very good response will provide useful information, while showing experience and knowledge within the category. The proposal is well thought out and addresses all requirements set forth in the RFP. The proposer provides insight into their expertise, knowledge, and understanding of the subject matter.

3 points Above Average

An above average response meets all the requirements and has demonstrated in a clear and concise manner a thorough knowledge and understanding of the subject matter and project. This response demonstrates an above average performance with no apparent deficiencies noted.

2 points Below Average

A below average response meets the requirements in an adequate manner. This response demonstrates an ability to comply with guidelines, parameters, and requirements with no additional information put forth by the proposer.

1 point Poor A poor response minimally meets most requirements set forth in the RFP. The proposer has demonstrated knowledge of the subject matter only.

0 points Unacceptable An unacceptable response does not meet the requirements set forth in the RFP. The proposer has not demonstrated knowledge of the subject matter.


2.2 Scoring Process (Steps 2 through 5) A. Initial scoring is done individually. Each evaluator is given an Excel workbook with individual vendor tabs to record their personal scores. These scores are for reference only. Once evaluators have completed their scoring, they return the summary scoring sheet to the OCVR Project Administrator for compilation. (See Appendix for Individual and Summary Score Sheets and Scoring Instructions) B. The Security Plan and Customer References are reviewed by smaller teams who will report their findings and scoring recommendations to the larger RFP Evaluation Team. C. The RFP Evaluation Team will discuss each scored item for individual vendors to reach an initial consensus score. The consensus scores will be grouped into larger categories where individual requirements may be weighted per team discussions. The final groupedconsensus scores carries over to the Final Scoring Sheet. Consensus scores will be calculated as 0 to 5. These scores will be translated to section point scores, using the point allocations in the RFP. The RFP Evaluation Team will select the competitive range from the top scoring proposals. These vendors will move to the next round of scoring for presentations and demonstrations. D. Once the presentations and demonstrations have been completed, these individual scores will be discussed by the RFP Evaluation Team to reach consensus scores. These consensus scores will be calculated as 0 to 5. These scores will be translated to section point scores, using the point allocations in the RFP and added to the scores from #C. E. Cost proposals will be opened and scored with highest points allocated to lowest total cost. All other proposals will receive a percentage of the points relative to the amount they proposed as compared to the Lowest Total Price proposal. F. Points from #D and #E will be added together to calculate a final total so the state can select the Apparent Successful Proposer. 2.3 Completeness Review The Completeness Review is to determine that each proposer has provided the mandatory information. Incomplete proposals will be disqualified in accordance with the RFP. The Completeness Review takes place in each step of the evaluation. 2.4 Compliance Review The Compliance Review will analyze whether the proposals comply with the RFP’s requirements in Step 1 evaluation. In Steps 2-4 the Compliance Review will analyze whether the proposals adequately address the business, technical, process, experience, economic, and financial requirements in accordance with the RFP.


2.5 Technical Merit Review The Technical Merit Review takes place in Steps 2-4 of the evaluation and will be based on each proposal’s quality of response in relation to,

• Project management methods and deliverables • Planning and design methods and deliverables • Test plans and testing proposals • User involvement and training planning and methods • Implementation planning and implementation methods • Post implementation service and customer care proposals • Oral presentations and product demonstrations

2.6 Company and Experience Review The Company and Experience Review takes place in Steps 2-4 of the evaluation and will be based on each proposal’s quality of response in relation to,

• Available and credible references for like projects • Experience and knowledge of complex database management systems • Stability of company and its products • Oral presentations and product demonstrations

2.7 Economic and Financial Merit Review The Economic and Financial Merit Reviews will take place in Step 5 of the selection process once Steps 1, 2, 3, and 4 are complete including the allocation of points for Initial responsiveness to mandatory requirements High level and functional requirements Mandatory proposal requirements System security proposal Oral Presentations Demonstrations The economic and financial analysis will address whether the proposals adequately respond to the requirements of the RFP and the financial objectives of the project. The financial analysis of the cost proposals will assess the Total Cost of Ownership for any OCVR system proposed and the highest points will be awarded to the Lowest Total Price proposal. All other proposals will receive a percentage of the points relative to the amount they proposed as compared to the Lowest Total Price proposal.


3. Evaluation Team Roles and Responsibilities The OCVR RFP Evaluation Team represents project stakeholders from both state and county government. Team members will do individual preliminary analysis on each proposal before meeting as a full group to compile a final score for each proposal. There will be work teams supporting the evaluation process. (See 3.2 for their roles and responsibilities) 3.1 Role and Responsibilities of the Evaluation Team Chair Chair the RFP Evaluation Team meetings Support SOS staff in maintaining the integrity of the evaluation process Defend the Evaluation Team’s analysis, as necessary Support SOS staff in debriefing unsuccessful proposers, as necessary Cast the tie breaker vote for proposal selection, as necessary 3.2 Role and Responsibilities of the Evaluation Team and Work Team Members Ensure that the proposals comply with the requirements of the RFP Individually score each proposal objectively and on its own individual merits Individually score oral presentations and demonstrations objectively and on individual merits Provide subject matter expertise and/or first hand customer experience The mandatory requirements response team will do the initial proposal response evaluation in Step 1 and report exceptions to the OCVR Executive Project Team The reference team will report customer questionnaire responses and scores to the RFP Evaluation Team The security plan team will evaluate and make recommendations to the RFP Evaluation Team 3.3 Role and Responsibilities of the Project Team Members Ensure confidentiality and conflict of interest procedures are followed by the teams Manage the teams’ work including scheduling meetings, assigning tasks, assisting in the conduct of work, ensuring consistency, and communicating to stakeholders Coordinating the work team’s efforts with the RFP Evaluation Team Develop the reference interview instrument for the reference work team Working with SOS Purchasing and Contracts to finalize selection process Documenting the evaluation process Preparing the supporting analysis for the selection decision


4.0 Schedule of Evaluation Events

June 2003 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4

VENDOR PRE-BID CONFERENCE

5 6 7

8 9 10 11 12 13 14

15 16 17 18

19

20 21

22/29 23/30 24 25 26 PROPOSALS DUE by 2 p.m. Step 1 Evaluation-Mandatory Response Req. Team

27 Step 1 Evaluation-Mandatory Response Req. Team

28


July 2003

Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1

Step 2 Evaluation, Reference Checks, Security Plans

2 Step 2 Evaluation, Reference Checks, Security Plans


4 HOLIDAY

5

6 7 Step 2 Evaluation, Reference Checks, Security Plans





12

13 14 Step 2 Evaluation, Reference Checks, Security Plans

15 Step 2-Work Teams Report & Eval Team Score PSB Video Conf Room 8-5




19

20 21 Step 2-Work Teams Report & Eval Team Score PSB Video Conf Room 8-5

22 Complete Step 2: Eval Team Establish Competitive Range PSB Video Conf Room 8-5

23 SOS Purchasing to Notify Competitive Range

24 25 26


27 28 29 Protest Period Ends

30 Presentations and Demo’s


August 2003 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1

Presentations and Demo’s

2

3 4 Presentations and Demo’s


6 Steps 1-4 scores totaled-Eval Team Evaluation Team Mtg- TBD

7 Steps 1-5 scores totaled by SOS staff and reported to Evaluation Team

8 Select Successful Proposer Purchasing to notify apparent successful proposer

9

10 11 Contract Work

12 Contract Work

13 Contract Work

14 Contract Work

15 Post Selection Review Period Ends

16

17 18 Contract Negotiations

19 Contract Negotiations




23


24/31 25 Contract Negotiations


27 Contract Preparation

28 Contract Preparation

29 CONTRACT AWARDED

30

5.0 Evaluation Team Members and Advisors EVALUATION TEAM MEMBERS ADVISORS Paddy McGuire, Deputy Secretary of State, Chair Glenn Newkirk, Quality Assurance Contractor John Lindback, Elections Division Director Jeff Morgan, Contracts & Purchasing Mgr Scott Smith, SOS Information Systems Director Mickie Kemper, Contracts Administrator Julie Pearson, OCVR Project Executive John McCormick, Attorney General’s Office Barbara Conway, OCVR Project Administrator Carl Wieneke, State Quality Assurance Manager Al Davidson, Marion County Annette Newingham, Lane County Maria Iturriaga, Harney County Dana Jenkins, Lincoln County TECHNICAL ADVISORS Murray Paolo, CIO, Yamhill County Heather Myers-Falk, SOS-ISD Carl Cole, IS Analyst, Linn County Elections Walt Ryder, SOS-ISD Leonard Higgins, SOS-ISD Systems Development Manager Rene Salstrom, SOS-ISD Tami Detwyller, Elections Division Bob Zakes, SOS-ISD Rick Retzman, SOS-ISD Ramjit Ramakrishnan, DAS - State IS Security Manager MANDATORY REVIEW TEAM MEMBERS Jeff Morgan, SOS Contracts Barbara Conway, OCVR Project Administrator REFERENCE CHECK TEAM MEMBERS Brenda Bayes, SOS Elections


SECURITY PLAN REVIEW TEAM MEMBERS Scott Smith, SOS-ISD Leonard Higgins, SOS-ISD Carl Cole, Linn County Murray Paolo, Yamhill County

Oregon Central Voter Registration Project

News & Politics

Transcript of Oregon Central Voter Registration Project