INFORMATIONAL

  The Department of Education will be conducting audits of the
  Virginia State Assessment Program (VSAP) Stanford 9
  administration slated for the Fall of 1998.  A random sample
  of 10% of Virginia's schools will be audited.  There are two
  purposes for the audit:

   1)   to observe actual test administration procedures to
        see whether prescribed, standardized procedures are
        being followed; and

   2)   to gather information on ways to improve procedures
        to maximize efficiency while maintaining standardized
        procedures and test security.

  Harcourt Brace Educational Measurement has trained a cadre of
  auditors who will conduct the audits in Virginia's schools. 
  Auditors, many of whom had experience as auditors during the
  Spring 1998 administration of the Standards of Learning (SOL)
  Assessments, were recruited throughout Virginia using a
  temporary employment agency.  Minimal qualifications for
  auditors include a four-year college degree with a preference
  for retired educators. Graduate education students enrolled in
  advanced degree programs within the state have also been hired
  as auditors.  Auditor candidates cannot be employed by,or have
  any relationship with, school division personnel which might
  result in a conflict of interest.  To qualify to conduct
  administration audits, each auditor has completed at least one
  full day of training on the VSAP and on audit procedures.  The 
  training was conducted jointly by the Department of Education
  and Harcourt Brace Educational Measurement.

    We encourage you to share the enclosed materials with your
  staff:

          VSAP Administration Audit - Observation Checklist
          VSAP Administration Audit - Post-Test Examiner  
            Protocol 
          VSAP Administration Audit - Post-Test School Test                                        Coordinator (STC)
            Protocol
          VSAP Administration Audit - Post-Test DDOT Protocol
          VSAP Administration Audit - Evaluation of the
            Auditor and Audit Process 
          VSAP Administration Audit - Sample Letter of                                             Introduction
          VSAP Administration Audit - Audit Information Sheet

  While the audits themselves will be unannounced, it is not the
  intent of the Department to surprise anyone regarding what is
  to be observed or asked.  It is important that you share with
  your staff members and teachers the essence of the audits so
  that they are aware of the activity should their school be
  audited.

  As noted in the enclosed procedures, your Division Director of
  Testing (DDOT) will be notified by fax a day ahead of any
  audit to be conducted in the school division.  The intent of
  that notification is to let the central office staff know that
  an auditor(s) will be present in the school division. 
  However, the notification will not include the name of the
  school(s) to be audited.  The audit procedures call for the
  auditor to meet with the School Test Coordinator (STC) in the
  school to be audited, observe the administration of one of the
  VSAP subtests in one classroom, and conduct post-observation
  meetings with the observed Examiner and the STC.  The
  principal of the school is encouraged to participate in these
  meetings.  The auditor will also arrange to meet with the DDOT
  to discuss the audit that has been conducted.  In the event
  that more than one school in the division has been selected to
  be audited, a single interview will be conducted with the DDOT
  after all audits in the division have been completed.

  Auditors have been informed of the need to observe the
  protocol for visitors that is in effect for each school they
  visit.  Each auditor will carry a letter of introduction on
  the letterhead of Harcourt Brace Educational Measurement, and
  will present photo identification upon request.  Each auditor
  will also have a brief one-page audit explanation to provide
  school staff members who might want further detail.

  It is our intent that these visits be as unobtrusive as
  possible.  The protocols for the VSAP audit are modified
  versions of those field-tested in four schools during the
  Spring 1998 administration of the SOL Writing test.  Feedback
  from auditors, Examiners, STCs, and DDOTs subsequent to the
  Spring 1998 SOL audits has been used to make the VSAP audit
  protocols as efficient and useful as possible.

  While information from the audit will be discussed in the
  post-observation meetings with the Examiner, STC, and DDOT, a
  more formal report will be provided to the school division and
  to the Department within 30 days of the close of the final
  audit in the state.  Auditors will leave copies of all
  completed audit protocols with the STC and DDOT once the audit
  is concluded.  An audit evaluation form will also be left with
  each person involved in the audit process in each school.

  During audits of the Spring 1998 SOL administrations, we
  observed many useful and innovative practices; sharing of
  those practices has been a by-product of that audit.  We look
  forward to observing additional such practices during the
  upcoming VSAP audits.

  Thank you in advance for your cooperation in this activity. 
  Any questions you might have concerning the VSAP audit
  procedures can be directed to the Division of Assessment and
  Reporting at (804) 225-2102.



  PDS/CMH/atb

  Enclosures:  A hard copy of this memo and its attachments
               will be sent to the superintendent's office.