The image http://www.votetrustusa.org/images/votetrust-small2.jpg cannot be displayed, because it contains errors.

 

The nation's clearinghouse for election audit information!
State and Local Election Integrity Organizations
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Maryland
Michigan
Minnesota
Missouri
Montana
New Jersey
New Mexico
New York
North Carolina
Ohio
Oregon
Pennsylvania
South Carolina
Tennessee
Texas
Vermont
Virginia
Wisconsin
VoteTrustUSA does not speak on behalf of any of the listed organizations.
: mosShowVIMenu( $params ); break; } ?>

   
Around the States

New York State’s Voting Machine Certification Process PDF  | Print |  Email
By Bo Lipari, New Yorkers for Verified Voting   
December 15, 2006

Issues, Status and Projections for Voting Machine Testing

Adequately tested voting systems are a prerequisite for well run elections and to ensure public confidence in election results. When it is completed, the current process of testing voting systems will culminate in New York State’s four Election Commissioners deciding to approve, or “certify” those systems which meet the State’s regulations. Certification testing is ongoing and is proving to be a mixture of good and bad news.

 

On the plus side, New York State has a strict set of statutory and regulatory requirements which voting rights advocacy organizations fought hard to adopt and which set high standards for voting machines to meet. On the minus side, the State Board of Elections seems willing to compromise strict compliance with regulation in order to allow voting machines to qualify when they otherwise would not. While the State Board has frequently stated they want to do certification testing right rather than do it fast, the tendency has been to err on the side of speed rather than rigor.

 

Subcontractors Performing Certification Testing

 

Lacking required technical expertise of their own, the New York State Board of Elections has contracted out the work of machine certification testing to two contractors. The State Board provides nominal oversight, reviewing status reports and monitoring the schedule, but lacks the technical competence to meaningfully evaluate machine vendors and consultants. Unfortunately for New York State voters, the Board of Elections has consistently underestimated the complexity and scale of the task before them.


The principle contractor managing certification testing for the State Board of Elections is CIBER, Inc., one of three large testing companies which have close ties to the voting machine vendors. CIBER was responsible for the certification of several voting systems that were later shown to have defective software and defects, and were subsequently de-certified by several states. CIBER’s performance for New York State has thus far been poor, tending to favor machine vendors’ very loose interpretation of State requirements. Ciber has submitted inadequate drafts of Master Test and Security Test plans, and seems willing to tolerate poor testing practices in order to allow machines to pass tests they otherwise would not.


New York State regulations call for an independent security review of voting machine source code. The firm NYSTEC was hired to fulfill this role. NYSTEC was hired without any real evaluation by the State Board of Elections or voting rights advocacy organizations. Many, including the author, were concerned that NYSTEC was unqualified to perform the independent analysis as they lacked any relevant experience with voting systems. But to date NYSTEC has indeed acted independently of CIBER, pushing back on several important security issues. Still ahead of NYSTEC is the actual source code security review.


Can Current Voting Systems Meet State Requirements?

 

New York State’s ongoing process of voting machine certification is charting unexplored territory with strict statutory requirements beyond that called for in most other states. The current inability and/or unwillingness of the voting machine vendors to meet State requirements calls into question whether any of the systems currently being tested can be certified unless the State Board of Elections compromises on the regulations. Unfortunately, current indications are that the State Board may be willing to sacrifice full compliance with the letter of New York State law in order to meet the currently targeted September 2007 implementation deadline.

• New York State’s 2005 Voluntary Voting System Guidelines Requirement
One of the strengths of New York State regulations is the requirement that all systems must meet the Federal 2005 Voluntary Voting System Guidelines (VVSG), the first state to do so. Initial reports indicate that all systems are failing tests which prove compliance with this high bar set by New York State. Voting systems which do fail to meet the 2005 VVSG requirements cannot be certified under State regulations. This raises the distinct possibility that, if the Board of Elections is strict in its interpretation of statute, no systems will pass certification testing, and New York State must delay HAVA implementation beyond the current September 2007 target.
• New York State’s COTS Software Escrow Requirements
New York State has a statutory requirement that requires that all source code be placed in escrow with the State, including so called “COTS” source code. COTS (Commercial Off The Shelf) refers to third party software developed by other companies used in the voting machines. One example of COTS software is the operating system, Microsoft Windows XP, used by the Sequoia and Avante DREs currently being tested. Machine vendors and CIBER want to interpret the State’s regulations in the loosest fashion, claiming that they cannot and should not provide source code for COTS software. But exemptions for COTS software must not be compromised away, especially not in ways that decrease transparency and security.

 

If the State Board of Elections stands fast on the statutory COTS requirement, it is likely that at least two DRE systems currently undergoing testing will not be certified. But there is danger the Board may negotiate a compromise, using a loose interpretation of the statute which weakens security and transparency protections.

Problems in CIBER’s Testing Practices

 

Voting machine vendors must at a minimum be held to the same development standards used for commercial software available to consumers. But several practices used by CIBER call into question the rigor of the certification process and how much leeway is being given to voting machine vendors, allowing them to pass tests they would not otherwise survive.

• Machine vendors have been allowed to submit software changes to products during testing.
It was reported at a State Board of Elections meeting in November that machine vendors have continued to submit changes to products during certification testing. This practice makes it virtually impossible to conduct rigorous testing, is contrary to standard software development industry practice and reveals that the vendors have submitted immature products that cannot meet State standards without numerous and ongoing changes. 
• Repeating Failed Tests Until They Pass
Early reports indicate CIBER uses an ill advised procedure—failed unit tests are repeated until there is a single case when the defect does not occur, and then the test is marked as passed. This procedure is a dangerous practice that will result in severe machine failures on Election Day.

Many software defects are not 100% repeatable - they may occur 3 times in a row but then not occur again for a time due to changed internal conditions. But because a test passes once after several tries does not mean the defect will not manifest itself in actual operation—it most assuredly will.

Looking Forward

 
According to the most recent schedule published by the State Board of Elections , voting systems are scheduled to be certified on February 20, 2007. Certification testing is still ongoing, with a significant portion of the work still to be completed. Following is a brief discussion of some of the issues that still lie ahead of us.

• Possibility of Further Delays
The certification schedule was recently delayed when a review found CIBER’s security test plan had significant shortcomings and was inadequate. A delay caused by a rigorous testing process is correct and necessary and should be applauded, not discouraged. There must be a willingness to accept further delays when they result in a more thorough review of systems.
Other reasons for delay, noted above, are the difficulty voting machine vendors are having meeting State requirements during testing.
• Is the State Board of Elections rushing to complete certification and compromising rigor?
One concern of voting rights advocacy organizations is the willingness on the part of some Board of Elections officials and staff to compromise regulatory requirements in order to complete testing on schedule. If too much is conceded, New York State could find itself certifying voting systems which fail to function properly during elections, are difficult to use and maintain, and that must be replaced within a few years.
 
• The State Board of Elections is unprepared for the next steps in voting machine adoption.
The Board has consistently underestimated the complexity and scale of the task of testing, certifying, and deploying new voting systems. They are out of their depth both technically and in their ability to manage a project of this size and importance. 

After certification, there is still much that needs to be done that the New York State Board of Elections has not yet even begun to address. Crucial procedures and standards for acceptance testing, software version verification, chain of custody issues, and a host of other items have yet to be even considered. These procedures are crucial to the proper use and roll out of new computerized voting machines across the State, and cannot be simply made up as we go along.


New York State voters have a right to the best voting systems, testing, standards and procedures. The State Board of Elections, through strict interpretation of state Election Law and regulations and a willingness to reject voting systems which do not meet those requirements, must provide that guarantee.

Comment on This Article
You must login to leave comments...
Other Visitors Comments
You must login to see comments...
< Prev   Next >
State Resources
Election Law @ Moritz
Electionline
National Conference of State Legislatures
Verified Voting
Model Legislation
: mosShowVIMenu( $params ); break; } ?>
State Pages
Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
District of Columbia
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
Nebraska
Nevada
New Jersey
New Hampshire
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
Guam
Puerto Rico
: mosShowVIMenu( $params ); break; } ?>