COMMONWEALTH OF VIRGINIA
DEPARTMENT OF EDUCATION
P.O. BOX 2120
RICHMOND, VIRGINIA 23218-2120
SUPTS. MEMO NO. 126
SEPTEMBER 7, 2001

INFORMATIONAL
 
TO: Division Superintendents
FROM: Jo Lynne DeMary
Superintendent of Public Instruction
SUBJECT: Virginia State Assessment Program Administration Audit Information

The Department of Education again will conduct audits of the Virginia State Assessment Program (VSAP) Stanford 9 administration scheduled this year. A random sample of 10 percent of Virginia's schools will be audited. There are two purposes for the audit:

1) to observe actual test administration procedures to confirm whether prescribed, standardized procedures are being followed; and,

2) to gather information on ways to improve procedures to maximize efficiency while maintaining standardized procedures and test security.

 Harcourt Educational Measurement has trained a cadre of auditors who will conduct the audits in Virginia's schools. Auditors, many of whom had experience as auditors during recent administrations of both VSAP and the Standards of Learning (SOL) Assessments, have been recruited throughout Virginia. Each auditor has a four-year college degree, and preference in hiring auditors was given to retired educators; graduate education students enrolled in advanced degree programs within the state also have been hired as auditors. Auditors cannot be employed by or have any relationship with personnel that might result in a conflict of interest. To qualify to conduct administration audits, each auditor has completed at least one full day of training by the Department of Education and Harcourt Educational Measurement on the VSAP and audit procedures.

Attachments to this memo include:
VSAP Administration Audit - Sample Letter of Introduction
VSAP Administration Audit - Audit Information Sheet
VSAP Administration Audit - Observation Checklist
VSAP Administration Audit - Post-Test Examiner Protocol
VSAP Administration Audit - Post-Test School Test Coordinator (STC) Protocol
VSAP Administration Audit - Post-Test DDOT Protocol
VSAP Administration Audit - Evaluation of the Auditor and Audit Process

While the audits will be unannounced, the Department of Education does not intend to surprise anyone regarding what is to be observed or asked, so we encourage you to share the attached materials with your staff. It is important that you also share information about the audits with your staff members and teachers so that they are aware of the activity should their schools be audited. As noted in the attached procedures, your Division Director of Testing (DDOT) will be notified by fax the day prior to any audit to be conducted in the school division. The intent of that notification is to let the central office staff know that an auditor or auditors will be present in the school division. However, the notification will not include the name of the school to be audited.

The audit procedures call for the auditor to meet with the School Test Coordinator (STC) in the school to be audited, observe the administration of one of the VSAP subtests in one classroom, and conduct post-observation meetings with the observed Examiner and the STC. The principal of the school is encouraged to participate in these meetings. The auditor also will arrange to meet with the DDOT to discuss the audit that has been conducted. In the event that more than one school in the division has been selected for audit, a single interview will be conducted with the DDOT after all audits in the division have been completed.

Auditors have been informed of the need to observe the protocol for visitors that is in effect for each school they visit. Each auditor will carry a letter of introduction on the letterhead of Harcourt Educational Measurement and will present photo identification upon request. Each auditor also will have a brief one-page audit information sheet for school staff members who might want additional details. It is our intent that the audits be as unobtrusive, efficient, and useful as possible.

While information from the audit will be discussed in the post-observation meetings with the Examiner, STC, and DDOT, a more formal report will be provided to the school division and to the department within 30 days of the close of the final audit in the state. Auditors will leave copies of all completed audit protocols with the STC and DDOT after the audit is concluded. An audit evaluation form also will be left with each person involved in the audit process in each school.

During audits of previous VSAP and SOL administrations, we have observed many useful and innovative practices. Sharing those practices has been a by-product of the audits, and we look forward to observing such practices during the upcoming VSAP audits.

Thank you in advance for your cooperation in this activity. If you have any questions or need additional information, please contact Scott Taylor (mailto:jtaylor@mail.vak12ed.edu), specialist for Assessment at (804) 225-2102.

JLD/SLR/JST/jc

Attachments

/administrators/superintendents_memos/2001/inf126a.pdf