WHAT IS A PROPER SOURCE CODE REVIEW?

Attachment to the Joint Statement
WHAT IS A PROPER SOURCE CODE REVIEW?

By AES Watch
Feb. 22, 2010

The review of the source code of the Automated Election System by interested political parties and groups is a right enshrined in RA9369. A proper source code review is one that is conducted comprehensively following IT industry best practices and accepted methodologies in software quality and integrity assurance and using automated tools for testing parts or the whole of the code.

Purpose of the review: The purpose of the review is to check for conformity of the AES 2010 computer programs (EMS, PCOS, CCS, and related utilities) to the provisions of RA9369 and related Philippine election laws, the COMELEC Terms of Reference to Bidders 2009, the COMELEC General Instructions and related COMELEC implementing rules and regulations, and for conformity to US EAC 2005 VVSG, wherever this US standard does not conflict with Philippine elections laws and regulations.

Basis for the review: The review must be based on a set of standards. In the case of the AES, we propose the use of the EAC VVSG 2005, the Voluntary Voting System Guidelines adopted by the Election Assistance Commission of the United States in December 2005. The purpose of the VVSG is “to provide a set of specifications and requirements against which voting systems can be tested to determine if the systems provide all of the basic functionality, accessibility and security capabilities required of voting systems.” The same VVSG was presumably used by SysTest Labs for the review and certification it performed.

Coverage of the Review: The review must cover the source codes of all the programs running the various components of the AES, including but not limited to the source codes of the Election Management System (EMS), the Consolidation and Canvassing System (CCS), Election Event Designer and Precinct Count Optical Scanner (PCOS), Election Programming Station (EPS), and other components, including the operating system in the PCOS (customized cLinux as disclosed in the Smartmatic’s SAES product description), the signing software, and transmission software, and the encryption software.

Format of the Source Code: The source code must be made available in editable format, which may be used with document or programming editing tools and/or automated software testing tools. COMELEC must provide a copy of the baseline source code and of the customized source code of all the AES software components.

Required Documentation : The reviewers must be provided all existing documentation related to the AES, including comments embedded within the source code, preferably comments in each module explaining what the particular module does. The documents that must be supplied by the Comelec and Smartmatic-TIM include but are not limited to: (a) Architectural Overview of the software, (b) Code Overview, how it is structured, its logic and interfaces, (c) Functional Summary. The user and administration manuals of the AES and its components must also be provided. The Comelec must also provide the specifications document that led to the customization of the AES. A more exhaustive list of required documentation may be found in EAC VVSG 2005. In addition, the SysTest Labs report must be shared with the reviewers, as well as its test procedures and test results.

Test Facility and Environment: The review may be conducted in a controlled facility designated by Comelec but it must be a comfortable area with ample workspace large enough to accommodate at least ten reviewers at any one time and open beyond normal office hours each day. Comelec must also provide the reviewers with a sufficient number of computer workstations and/or notebooks with printers as well as the software of automated tools needed for the review.