SUPTS. MEMO. NO. 45
March 5, 1999
|FROM:||Paul D. Stapleton
Superintendent of Public Instruction
|SUBJECT:||Standards of Learning Administration Audit Information|
The Department of Education will be conducting audits of the administration of the Standards of Learning (SOL) tests slated to be administered in the spring of 1999. A random sample of 10 percent of Virginia's schools will be audited. There are two purposes for the audit: 1) to observe actual test administration procedures to see if prescribed procedures are being followed; and, 2) to gather information on ways to improve procedures to maximize efficiency while maintaining security and standard procedures. Harcourt Brace Educational Measurement has trained a cadre of auditors who will conduct the audits in Virginia's schools. Auditors were recruited throughout Virginia using a temporary employment agency. Minimal qualifications for auditors include a four-year college degree and preference for retired educators. Graduate education students enrolled in an advanced degree program within the state may be considered. Auditor candidates cannot be currently employed by or have a relationship with personnel which might result in a conflict of interest. To qualify to conduct administration audits, each auditor has completed a full day of training on the assessment program and the audit procedures. The training was conducted jointly by the Department of Education and Harcourt Brace Educational Measurement. Attachments to this memo include: * Spring 1999 SOL Writing Test, Observation Checklist * Spring 1999 SOL Writing Test, Post-Test Examiner Protocol * Spring 1999 SOL Writing Test, Post-Test School Test Coordinator(STC) Protocol * Spring 1999 SOL Writing Test, Post-Test Division Director of Testing (DDOT) Protocol * Spring 1999 SOL Writing Test, Audit Evaluation * Spring 1999 SOL Assessment Auditor Sample Letter of Introduction We encourage you to share these materials with your staff. While the audits themselves will be unannounced, it is not the intent of the Department of Education to surprise anyone regarding what is to be observed or asked. It is important that you share with your staff members and teachers the essence of the audits so that they are aware of the activity should their school be audited. As noted in the attached procedures, your DDOT will be notified by fax a day ahead of any audit to be conducted in the school division. The intent of this notification is to let the central office staff know that an auditor will be present in the school division. However, the notification will not include the names of the schools to be audited. The audit procedures call for the auditor to meet with the STC in the school to be audited, observe one classroom during test administration, and conduct a post-observation meeting with the Examiner observed and the STC. The principal of the school is encouraged to participate in the exit interview. Auditors will also arrange to meet with the DDOT to discuss the audits conducted. In the event that more than one school is being audited in a school division, a single exit interview will be conducted with the DDOT. Auditors have been informed of the need to observe the protocol for visitors that is in effect for each school they visit. Each auditor will carry a letter of introduction on Harcourt Brace Educational Measurement stationery and will have photo identification available upon request. Each auditor will also have a brief one-page audit explanation to provide staff members who might want further details. It is our intent that these visits be as unobtrusive as possible. The protocols for the audit were field tested in four school divisions during the writing administration. Feedback from this field test was used to make the protocols as efficient and meaningful as possible. While information from the audit will be discussed in the exit interview, a more formal report will be provided to each school division and the Department of Education within 30 days of the close of the last audit. Auditors will leave copies of all audit documents with the school and school division after the audit is concluded. An audit evaluation form will also be left with each person involved in the process in each school. We look forward to observing many good practices across Virginia as the SOL tests are being administered. The sharing of these good practices will hopefully be a by-product of the audit process. We thank you in advance for your cooperation in this activity. Any questions you might have concerning the administration audit procedures can be directed to the staff of the Division of Assessment and Reporting at (804) 225-2102. PDS/jc Attachments: A hard copy of this memo and its attach- ment will be sent to the superintendent's office.