Overview
Introduction
Test security continues to be a critical issue for many states and districts. This applies not only to general content assessments used for accountability but also to states’ alternate assessments based on alternate academic achievement standards (AA-AAAS).
State tests must have integrity and the results must be valid (U.S. Department of Education, 2018). Fair and equitable assessment approaches must be provided to all students, including students with disabilities. This includes designing tests that are accessible, as well as the availability and use of accessibility features and accommodations for students who need them. Administering assessments fairly and equitably requires that states have good security policies and procedures.
As has been noted many times over the years, cheating and test piracy (i.e., stealing of test forms or items) pose major threats to the validity of test score interpretation and the credibility of large-scale assessment programs (e.g., Cizek, 1999; Wollack & Fremer, 2013). To ensure the soundness of test results, it is essential to devote systematic and productive efforts in all aspects of the design, development, administration, analysis, and reporting of the assessments. In recent years, most, if not all, states have taken additional steps to prevent cheating from occurring as well as to detect whether it has occurred. Many states have issued new policies and implemented stronger procedures to address security issues.
The security of accommodated general assessments as well as AA-AAAS is receiving increased attention. There is a need for thoughtful consideration of the ways in which possible test security measures may affect accessibility because there is a risk that some policies may adversely affect accessibility for some students with disabilities. Additionally, some accommodations are administered in a small group or individual setting, which may present unique test security issues. Other accommodations that some students with disabilities need to meaningfully access a test may require someone other than the student to also see the test (e.g., sign language interpreter, reader, scribe). Also, there sometimes is a need for test administrators or accommodations providers to access a test prior to test day so that adjustments can be made to test materials (for example, to enlarge text for students with low vision, or for sign language interpreters to have time to study materials to ensure appropriate sign language interpretation during the test). All these accommodations have potential security risks.
There is a need to update what we know about how students with disabilities and the tests they take are included in states’ test security policies and procedures. The previous analysis by Lazarus et al. (2014) provided a snapshot of how accommodated tests, alternate assessments, and other issues related to students with disabilities were addressed in states’ 2013-14 test security policies. The key findings from that analysis included:
- Almost all states had policies about whether an assessment could be duplicated or copied.
- The test security policies of about half of the states indicated that test administrators must receive training that includes information about accommodations.
- Many states’ policies addressed security issues surrounding the provision of accommodations that required a human to administer them. Thirty-four states had policies for transcription and scribing, 32 states had policies for human readers, and 30 states had policies for sign language interpretation.
- Other accommodations addressed in test security policies included: text to speech, speech to text, braille, extended time, and multiple days.
- Some policies mentioned small group and individual administration, though only a few provided details about how these types of administrations should take place to minimize the security risk.
- About half of the states had test security policies that specifically addressed the AA-AAAS.
This report provides an undated snapshot of how accommodated tests, alternate assessments, and related issues for students with disabilities were addressed in states’ test security policies in 2020-21. It is an update to the analysis of 2013-14 test security policies (Lazarus et al., 2014).
Test Security Guidelines and Professional Standards
Several documents that address test security and students with disabilities are highlighted here.
U.S. Department of Education Peer Review Guidance (Updated 2018). The purpose of the Department’s peer review of state assessment systems is to support states in meeting statutory and regulatory requirements for implementing valid and reliable state assessment systems under Title I of the Elementary and Secondary Education Act (ESEA). As part of the updated requirements for peer review, the U.S. Department of Education requires states to address and document their test security policies and procedures, including ones to prevent test irregularities and ensure the integrity of test results. This peer review guidance also strengthened requirements for assessing students with disabilities, such as ensuring that its assessments are accessible to students with disabilities, and that there is monitoring of test administration for both general and alternate assessments.
Among the Critical Elements in the peer review guidance that are relevant to this study are:
- 2.4 – Monitoring Assessments
- 2.5 –Test Security
- 5.4 – Monitoring Alternate Assessments
Standards for Educational and Psychological Testing (2014). Another source of guidance is the latest edition of Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014). This document, referred to as the Joint Standards, represents the gold standard in guidance on testing in the U.S. and worldwide.
Although the topic of test security does not have a separate chapter in the Joint Standards, it is addressed in a number of places. Most relevant is the inclusion of test security in the treatment of test design, administration, scoring, and reporting.
TILSA Test Security Guidebook (2013) and TILSA Test Security: Lessons Learned by State Assessment Programs in Preventing, Detecting, and Investigating Test Security Irregularities (2015). Both of these resources were developed by the Technical Issues on Large Scale Assessment (TILSA) Collaborative and published by the Council of Chief State School Officers. In the Test Security Guidebook (Olson & Fremer, 2013), one of the areas identified as having a risk of a test security breach was accommodations being used inappropriately to cheat. Although the Guidebook also included recommendations about how states could improve test security and reduce cheating, none specifically addressed accommodations or students with disabilities.
The other CCSSO resource, TILSA Test Security: Lessons Learned by State Assessment Programs in Preventing, Detecting, and Investigating Test Security Irregularities (Fremer & Olson, 2015), was developed to help states improve their existing test security policies, procedures, and protocols, and to implement enhanced approaches for the security of their state assessment programs. Similar to the Guidebook, this resource addressed several topics that could be applicable to accommodations and other topics relevant to students with disabilities (e.g., prevention, detection, follow-up), but students with disabilities were not explicitly addressed.
States’ Test Security Policies and Procedures (2013). This report by the U.S. Government Accounting Office (GAO) provides information from a national survey of state testing directors. States reported that their policies and procedures included 50 percent or more of the leading practices to prevent test irregularities in the following five areas: security plans, security training, security breaches, test administration, and protecting secure materials. Of the 28 states that administered computer-based assessments, the majority reported including half or more of the leading practices in computer-based testing. States also reported using various tools, such as statistical analyses of student data, monitoring, and audits of testing procedures, to oversee districts’ implementation of test security policies and procedures, and most states have used this oversight to identify cheating in recent years. States also identified areas where additional assistance with test security would be useful. The majority of states reported that it would be very or extremely useful if the U.S. Department of Education gathered and disseminated information on best practices in test security.
Operational Best Practices for Statewide Large-Scale Assessment Programs (2013). This document by CCSSO and the Association for Test Publishers (ATP) addressed “technology-based assessments” that are delivered or taken by computer, including “online” assessments, as well as those that are cached or remotely handled. It addressed new problems in program design, new practices for test security, and the need to handle test accommodations for special populations. One chapter discussed best practices for achieving test security throughout the assessment process of all paper-based and technology-based assessments, using methods that support the accessibility needs of students with disabilities and English learners.
Handbook of Test Security (2013). The Handbook of Test Security (Wollack & Fremer, 2013) was the first to offer insights from experts within and across the testing community, psychometricians, and policymakers to identify and develop best practice guidelines for the design of test security systems for a variety of testing genres. It provided insight into the prevalence of cheating and best practices for designing security plans, training personnel, and detecting and investigating misconduct, to help develop more secure testing systems and reduce the likelihood of future security breaches. Still, it had little to say about test security in regards to students with disabilities.
Considerations When Including Students with Disabilities in Test Security Policies (2014). This NCEO brief (Lazarus & Thurlow, 2014) noted that many states are concerned their test security policies may not adequately address accommodated tests, alternate assessments, and related issues for students with disabilities. The brief highlighted considerations for states to better address test administration procedures, accommodations, and other issues related to students with disabilities to help ensure the integrity and validity of a test.
Restart & Recovery: Meeting the Assessment Needs of Students with Disabilities During Times of Interrupted Schooling (2021). This report from CCSSO (Lazarus et al., 2021) provided recommendations for state departments of education on how to meet the assessment needs of students with disabilities, including English learners with disabilities and students with the most significant cognitive disabilities, during times of interrupted education due to extended school closures. It also included ideas for information that state departments can give to local schools.
Purpose of This Policy Analysis
This report provides an update to the information that was gathered in the report on 2013-14 policies. For this study, the guiding research questions were:
- What information do states’ 2020-21 test security policies contain about students with disabilities and the assessments they take?
- How are accommodations and accessibility features included in states’ test security policies?
- How do state policies for alternate assessments address security for these tests?
- What new resources have states developed to guide their best practice in conducting secure and fair assessments for students with disabilities?
Procedures
The information analyzed in this report was gathered from publicly available written documents on state department of education websites of the 50 states and the District of Columbia (DC). State documents were obtained between September and December 2021. The process included an analysis of documents related to the general assessment and the AA-AAAS.
The types of documents that addressed test security and confidentiality and students with disabilities varied across states. They included, for example, state policies on test security, test security manuals or plans, procedures or test administration manuals, general test security documents, accommodations/accessibility manuals, technical reports, and training materials. The number of publicly available documents that included information about test security and students with disabilities ranged across states from just a few to 11 documents. In some states, information about issues related to students with disabilities was in the state’s test administration manual (TAM). In other states, the information was in the accommodations or accessibility manual or test coordinator manuals (TCMs). In many states, the information was in both.
In a number of states, a new type of document, often called a Test Security Manual or Handbook, a Test Security Plan, or something similar, was found. These were reviewed closely for information on test security as it applied to students with disabilities, accommodations, and alternate assessments. In addition, new security policies requiring test administrators, proctors, accommodations providers, or others to sign affirmation forms or confidentiality agreements were also noted in many states. If found, this new information was also included in this study.
The search criteria and process conducted to gather information for each state used the following steps:
- Look at the list of data sources in the 2013-14 report (Lazarus et al., 2014) to identify documents the state had at that time, and then conduct a web search for any updated (more recent) versions of those documents. All relevant documents were included in this analysis.
- Search for security and test administration manuals. Those found to contain information about students with disabilities and accommodations were included in analyses.
- Search for each state’s accessibility or accommodations manual. Those found to contain information about test security and accommodations were included in analyses
- Search for any other documents that might have information about security and disability or accommodations. Those found were included in analyses.
Documents from state assessment consortia (e.g., Smarter Balanced, PARCC/New Meridian, Dynamic Learning Maps - DLM, etc.) that were accessed via links on state department of education websites were included in this analysis.
Appendix A includes a complete list of documents reviewed for this analysis along with links to the materials. All information in these documents that addressed students with disabilities and test security was compiled and then coded. The coding categories used for this study were similar to those used in the report on 2013-14 policies, although several additional themes were identified in this analysis. New codes were created for new themes so the data about them could be tabulated and reported.
Results
Characteristics of State Policies
As found in the 2013-14 policies report (Lazarus et al., 2014), there was wide variation across states in the ways in which accommodated tests, students with disabilities, and the assessments they take, were included in state test security policies. Some states provided extensive detailed information, while other states provided brief, broad statements, and still others provided little or no information.
As shown in Figure 1, states’ test security policies continued to address a number of issues. For additional details and specifications, see Tables B1 and B2 in Appendix B.
Figure 1. Number of States’ Test Security Policies That Addressed Selected Issues (N=51)
Security of test materials used for accommodated tests. The policies of 44 states specifically addressed security issues related to test materials used for accommodated tests. For example, the policy in Illinois stated:
Testing materials irregularities include removing secure test materials from the school’s campus or removing them from locked storage for any purpose other than administering the test. The District Test Coordinator must ensure that the following test materials are securely destroyed/shredded immediately after all testing is complete (either by the district or school). All accommodated responses (Do NOT destroy test booklets, answer documents, and Human Reader scripts.) Any original student responses that were printed from an assistive technology device or recorded in another accommodation document such as blank paper must be securely destroyed.
In Indiana, the security policy on test materials included:
Before each test administration, materials must be distributed and stored according to the Indiana Assessments Policy Manual. As described therein, testing materials must be secured at all times during test administration, including all breaks in the testing sequence. All read-aloud scripts, assessment books (used and unused), answer documents, test tickets, scratch paper, glossary sheets (ILEARN paper-and-pencil only) and online test access codes must be counted, reconciled, and returned to a centrally-located, locked, and secured area immediately upon the completion of each daily testing session.
All accommodated assessment books (regular print, large print, braille, and Spanish) were treated as secure documents, and processes were in place to protect them from loss, theft, and reproduction of any kind.
Role of IEP. Test security policies of 43 states contained information about the role of IEPs in determining students’ accommodations needs. IEPs were mentioned in test security policies, with varying amounts of explanation provided on how security was to be addressed in them. For example, Alaska’s test security policy and participation guidelines stated:
Next, the IEP or 504 Team should consider questions that relate to whether the assessment could still be administered: Would use of the adaptation cause a breach of test security? Before rejecting an adaptation for security reasons, an IEP or 504 Team member or other school or district official should consult with the department. In special cases, security can be bolstered to accommodate special needs.
The policies of a few states addressed how test validity would be affected if a student did not receive the accommodations on his or her IEP. For example, the California policy said:
Approval of an unlisted resource that has not been pre-identified will be granted by the CDE on the basis of the IEP team’s and/or Section 504 plan’s designation and if the unlisted resource does not compromise the test’s security. The CDE shall make a determination of whether the requested unlisted resource changes the construct being measured. IEP teams should not allow the impact of an LEA’s accountability to outweigh the needs of the student….
Training requirements related to accommodations. The test security policies of 41 states indicated that training was required that specifically addressed accommodations or students with disabilities for all test administrators and accommodations providers. Kansas is an example of a state that required everyone involved in testing to receive training. The state’s policy, as shown in a PowerPoint slide used for training of district and building-level personnel, indicated:
Train building-level personnel before local testing begins; include training regarding test security procedures, ethics of testing, and reporting/documentation of accommodations. Building-level personnel includes any staff member who administers a state assessment, such as administrators, educators, and para-educators. Parents and school volunteers may not administer a state assessment.
District Test Coordinator Responsibilities: Train district and building-level personnel before testing begins, including training regarding test security procedures, ethics of testing, and reporting and documentation of accommodations.
Kentucky is another state with a policy that specifically addressed the proper training of accommodations providers:
Any individual providing accommodations for students in special populations must also be trained on the Inclusion of Special Populations in the State Required Assessment and Accountability Programs (703 KAR 5:070).
Duplication and copying of test materials. The policies of 41 states addressed security for the duplication and copying of test materials. Most states do not allow duplication or copying. For example, Georgia’s Assessment Administration Protocol Manual stated:
Test content must not be copied or distributed. All statewide mandated tests given in Georgia are secure. Test items, student responses, and/or answer documents that are copied (by hand or by photocopying) or distributed violate test security and render the results of the test invalid.
A few states may allow duplication or copying under certain circumstances. The Texas policy for its AA-AAAS indicated:
In order to access some allowable accommodations, it may be necessary to photocopy secure materials. These allowable accommodations must be documented in the student’s IEP. Many of the techniques used to make images more accessible require methods that might distort the images on the next page of the student test booklet. In these instances, photocopying of the student test booklet may be required. Adhere to these guidelines when photocopying the student test booklet. The student test booklet cannot be disassembled. The district must maintain test security and confidentiality when photocopying the student test booklet.
Several states had policies for handling secure test materials that applied to a variety of activities, including duplication or copying of test materials, for both the general and alternate assessments. The policy in Vermont specified that:
Breaches of test security include but are not limited to copying of test materials, failing to return test materials, coaching students, giving students answers, and/or changing students’ answers. Unless needed as a print-on-demand or braille accommodation, no copies of the test items, stimuli, reading passages, performance task materials, or writing prompts may be made or otherwise retained.
Qualifications of test administrators and accommodations providers. Test administrators and test proctors need to know how to appropriately administer accommodations to students with disabilities without compromising test security. Because some accommodations are provided by a human access assistant (e.g., sign language interpreter, human reader, scribe), security must be addressed for these situations. In 2020-21 test security policies, 36 states addressed who may, or may not, provide accommodations. For example, Georgia had the following policy in place for test examiners and proctors:
Test examiners must be certified educators by the Georgia Professional Standards Commission (GaPSC) and must be assisted by proctors, when required. It is highly recommended that proctors be system employees; however, it is permissible to allow volunteers to serve as proctors, provided they receive all required training. Parents, other relatives, and guardians must not proctor the class or grade level in which their child or a relative is a student. Parents who are school employees should not serve as the examiner for their child or a child of a close relative. Further, it is highly advisable that, if possible, parents should avoid serving as an examiner or proctor for the grade level(s) of their children.
Hawaii’s policy for test administrators and accommodations providers for 2020-21 was:
The test reader should be an adult who is familiar with the student, and who is typically responsible for providing this support during educational instruction and assessments. Scribes must be DOE certificated employees who have Smarter Balanced test administrator certification. The scribe should be an adult who is familiar with the student, such as the teacher or teaching assistant who is typically responsible for scribing during educational instruction and assessments.
Hawaii also had a policy that specifically addressed its alternate assessment:
The HSA-Alt must be administered by a certified Test Administrator, who will most likely be the student’s classroom teacher. Classroom aides or paraprofessionals may not administer the HSA-Alt.
The Hawaii policy also provided a list of other school personnel (e.g., general educator, school counselor, long-term substitute teacher) who might administer HSA-Alt if they meet specific criteria.
In Idaho, the policy was:
Test Administrators and Observers must not have a conflict of interest or the appearance of a conflict of interest. For example, Test Administrators and Observers should not be present in rooms in which a child who resides in their household is testing. Test Administrators and Observers may not use their knowledge of test content to violate the integrity of the assessments through acts of coaching or other prohibited actions. Teachers can be Observers for emotional and behavioral support in rooms where their students are testing. To minimize student interactions that can lead to additional COVID-19 spread, teachers this year may act as Test Administrators for both in-person and remote administrations of their own students.
Changes to standard test procedures. The test security policies of 29 states described which changes to standard procedures were allowed or not allowed. For example, Maine’s policy clarified those instances when a change was allowable:
Some students may require a unique assessment environment as an approved accommodation. In such cases, the accommodation overrides the general assessment environment requirements.
However, some states policies indicated that standard security procedures could rarely, if ever, be changed. For example, Ohio’s policy stated:
Providing a student with modifications during Ohio’s State Tests may constitute a test irregularity and will result in an invalidated score (the score will not be counted) and/or an investigation by the state into the school’s or district’s testing practices. Moreover, providing modifications to students during statewide tests may have the unintended consequence of reducing their opportunities to learn critical content and may result in adverse effects on the students throughout their educational careers.
Personnel knowledgeable about special education requirements. The policies of 28 states specifically indicated that individuals involved in the administration of assessments to students with disabilities understand relevant special education requirements (e.g., IEPs, accommodations policies, participation guidelines, etc.). For example, Montana’s policy said:
Having an Assessment Team hierarchy not only forces a chain of custody, but also helps prevent one individual from being overwhelmed or responsible for all assessment matters. For example, persons most familiar with and experienced in student educational plans such as IEPs should be responsible for setting up individual student supports and accommodations on a case-by-case basis in the TIDE system.
Nebraska’s policy stated:
EL teams, IEP teams, and educators for 504 plans make decisions about accommodations… Decision makers should provide evidence of the need for accommodations and ensure that they are noted on the IEP or 504 plan, applicable. Decision makers are responsible for entering information on linguistic supports and/or accommodations from the IEP or 504 plan.
New Jersey provided information on its policies related to special education requirements, IEPs, and use of accommodations:
Test Administrators must be familiar with the student’s Individualized Education Plan (IEP) or 504 plan, and should know in advance which accommodations are required by the student, and for which test (ELA and/or Mathematics/Science) the student is designated to receive a human signer. Test Administrators must be aware of whether a student requires additional tools, devices, or adaptive equipment that has been approved for use during the test, such as a magnifier, closed circuit television (CCTV), abacus, brailler, slate, stylus, etc., and if use of these tools impacts the translation of the test, the signer should be made aware of this.
General information about security and accommodations. A slight majority of states (28) included general information about accommodations security in their test security policies for 2020-21. For example, the Hawaii policy stated:
The Hawaii State Department of Education requires that all schools closely monitor statewide assessments. Systematic delivery of the assessments with individualized, specific, appropriate accessibility features included is essential to the delivery of a fair, valid, and reliable assessment. As a first step in this test delivery oversight process, school test coordinators are asked to input (or request verification for) each student’s individual test accessibility feature(s). Provision of inappropriate accommodations, or any modification to the state assessment that somehow alters item construct or cognitive demand will invalidate results.
Other states’ policies included reminders of appropriate testing procedures and indicated that state accommodations policies should be followed. For example, North Carolina’s policy stated:
To ensure standardized testing procedures for students with disabilities who require test accommodations, administrative procedures must be developed and implemented to ensure individual student needs are met, and at the same time, to maintain sufficient uniformity of the test administration to maintain test validity and to fulfill the requirements of testing for accountability… Accommodations designated for the tests should be consistent with accommodations used routinely during classroom instruction and similar classroom assessments. It is vital for students with disabilities to receive accommodations on state-mandated tests that allow them to demonstrate their true abilities; however, students must not receive unnecessary, inappropriate, or unfamiliar accommodations.
Other Issues. Eighteen states also had security policies that addressed other issues. These included, for example, remote testing, logistics for accommodated assessments, use of assistive technology, and other new challenges that have received attention in recent years.
Additional Themes in 2020-21 State Policies
An extensive list of state resources and materials related to security for students with disabilities was identified during the search process. Many additional documents were found for 2020-21 than had been identified for 2013-14. Some of these were resources that addressed additional themes. Codes to identify these additional themes were developed. New themes included:
- State Had a Test Security Manual
- An Affirmation Document or Agreement Requiring Signature was Used
- A Confidentiality Agreement Requiring Signature was Used
- Documentation of Procedures for Irregularities (Peer Review Critical Element 2.5)
- Documentation of Monitoring Procedures (Peer Review Critical Elements 2.4 and 5.4)
Figure 2 shows the number of states that addressed these new themes in their 2020-21 documents. For additional details and specifications, see Tables B3 and B4 in Appendix B.
Figure 2. Number of States’ Test Security Policies That Addressed New Test Security Themes (N=51)
Test Security Manual. Quite a few states had developed new documents for their 2020-21 test security materials. Many had consolidated the state’s test security policies and procedures into a single stand-alone document, typically called a Test Security Manual (or Handbook). These documents typically brought together various state policies, procedures, and practices on test security into one report, along with information on security training, monitoring, and following up on irregularities. Often, a test security manual addressed activities related to prevention, detection, and follow-up investigations of test irregularities or incidents.
In the numerous 2020-21 test security documents that were collected for this study, we found that 17 states had such a resource that addressed many if not most of these things and could be considered a test security manual. In some states, the Test Security Manual only addressed requirements for the summative general assessment, while in other states this manual was inclusive of assessing students with disabilities and the use of test accommodations or addressed security for the AA-AAAS. Our analysis of 2020-21 documents indicated that 15 of the 17 states had policies for students with disabilities or AA-AAAS in their test security manuals. Few states had this type of resource in the analysis of 2013-14 policies.
Affirmation of Training Signature. In 2020-21 policies, we found that some states had developed documents and forms that required district and school staff to sign legal affirmations that they had been trained specifically on test security for students with disabilities, accommodations, or alternate assessments. The policies of seven states addressed this. For example, Georgia’s policy said:
STCs and SchTCs are instructed to clearly document attendance at all training sessions and include agendas and supporting presentations and/or handouts with specific guidance and policy related to test security and assessment administration protocols.
South Carolina had the following policy for its security affidavit:
TAs and TMs must validate the Test Administration Security Affidavit for each subject in which the student is assessed to confirm that all security procedures were followed during the assessment. Only the TA and TM validate the affidavit. Failure to complete the validation of the Test Administration Security Affidavit for each subject by May 14, 2021, will result in the test not being scored and scores not being reported for the student in the subject. There are no exceptions.
Confidentiality Agreement Signature. Many states had developed documents or forms that required district and school staff to sign confidentiality agreements or non-disclosure agreements (NDAs) on maintaining test security for students with disabilities, accommodations, or alternate assessments. These documents usually referred just to the summative assessment, although some were worded more generally for all the types of assessments the state administers. The policies of 22 states addressed these types of confidentiality agreements.
For example, Alaska had this policy for the security of its tests, with different levels of confidentiality agreements focusing on requirements for district personnel to maintain test security:
4 AAC 06.765 Test security; consequences of breach… [listing of state’s security requirements for all test materials and testing procedures – Districts must complete forms]
Level 1-2 Test Security Agreement (TSA) - District Level Oversight
[Note: State has 5 levels of security agreements for testing personnel. Levels 1 and 2 described above. Guidance covers all aspects of testing for key roles of district and school personnel with forms at each level to complete.]
In its policies, Texas required the following oaths on maintaining security and confidentiality:
Oaths of Security and Confidentiality. All district and campus personnel who participate in state-mandated testing or handle or have access to secure test materials must be trained and sign an oath of test security and confidentiality. Any person who has more than one testing role (for instance, a district coordinator who also serves as a test administrator) must receive appropriate training and sign a security oath for each role. Test security oaths are valid for the entire school year, including fall, spring, and summer testing, as well as any field testing and mandatory sampling conducted during this time period. Test administrators must complete the general oath. General oaths should be printed or saved and provided to the campus coordinator. All oaths are required to be maintained by the district for a period of five years. The general Oath of Test Security and Confidentiality is available in the Security section of the Coordinator Resources. Understand your obligations concerning test security and confidentiality….
Test Security Agreements Specifically for AA-AAAS. A few states had test security agreements designed specifically for TAs of AA-AAAS. For example, Virginia’s policy contained the following requirements for the confidentiality of the VAAP (its alternate assessment):
Confidentiality Agreement for Alternate Assessment: 2020–2021 School Division Personnel Test Security Agreement for the Virginia Alternate Assessment Program: All individuals who may be involved in the administration of the Virginia Alternate Assessment Program (VAAP) to include collection development, monitoring, and pre-scoring MUST read, understand, and agree to adhere to the following.
Colorado had the following policy on maintaining security for its AA-AAAS (CoAlt) and an additional requirement for confidentiality forms to be signed by all TAs:
Maintaining the security of all test materials is crucial to obtaining valid results from the CMAS and CoAlt: Math, ELA (including CSLA), Science, and Social Studies assessments. The security of all test materials must be maintained before, during, and after test administration…. Engaging in prohibited activities may result in an investigation, suppression of scores, and possible disciplinary action. This form must be signed by all individuals involved in the administration of the CMAS and/or CoAlt: Math, ELA, Science, and Social Studies assessments to certify that security measures will be maintained and that prohibited activities, such as the examples identified below, have been acknowledged and understood.
Iowa had the following statement about the use of a confidentiality form in its directions for TAs of the DLM AA-AAAS:
Iowa has a separate test security/confidentiality agreement for human readers, scribes, translators, and paraeducators.
Test administrators and assessment coordinators for DLM alternate assessment must agree to security agreement in Educator Portal.
Documentation of Procedures for Irregularities (Peer Review Critical Element 2.5). Critical Element 2.5 specifies that states must have policies and practices in place for the security of their assessments, including those used with students with disabilities, accommodations, or alternate assessments. States need to document their procedures for dealing with testing irregularities and security incidents. Not surprisingly, by 2020-21, the policies of 41 states addressed the documentation of procedures for irregularities. Most states had procedures and systems for documenting irregularities and following up on security incidents, and a few specifically addressed students with disabilities and accommodated tests. For example, Wisconsin’s policies stated:
Report all testing irregularities to the SAC [school assessment coordinator], including if a student cheating, or use of an unallowable accommodation. Situations that will lead to the invalidation of test scores include:
- A student was provided with an unallowable support or accommodation or an accommodation that was not in the student’s Individualized Education Program (IEP), 504 plan, or English Learners (EL) plan.
- A student was not provided an allowable support that they should have been provided or accommodation that was in the student’s IEP, 504 plan, or EL plan, and this lack of support or accommodation significantly affected the student’s performance.
Documentation of Monitoring Procedures (Peer Review Critical Elements 2.4 and 5.4). Peer review guidance requires states to monitor test administrations for both the general and alternate assessments and to document their monitoring procedures. To meet the requirements in Critical Elements 2.4 and 5.4 of the peer review guidance, states must submit detailed documentation of their process for randomly monitoring testing in schools. The policies of 38 states in 2020-21 addressed this, with most states having specific procedures for documenting their monitoring activities. For example, Massachusetts’s testing policy said:
School Observations. In order to ensure the security and proper administration of the MCAS program, the Department conducts announced and unannounced monitoring visits to schools to observe the procedures followed during test administration. This may include entering testing rooms to observe students and test administrators directly. Principals and designees should be prepared to meet observers upon their arrival and also be available during the observation to answer questions.
Test Security Issues Addressed in State Policies for AA-AAAS
Figure 3 shows how several key criteria for AA-AAAS were addressed in state policies. For details and specifications on these criteria, see Tables B5 and B6 in Appendix B.
Figure 3. Number of States’ Test Security Policies That Addressed AA-AAAS (N=51)
Security of assessment and materials. In their 2020-21 test security documents, 42 states had policies for the AA-AAAS that addressed the security of the assessment and the materials used to administer it (e.g., materials needed to administer performance tasks, materials for individualized accommodations, etc.). Most, if not all, states required their AA-AAAS to be secure regardless of the design of the assessment (e.g., selected response items, performance tasks, portfolio, checklist, etc.). For example, Hawaii’s policies for a secure AA-AAAS said:
The HSA-Alt test materials, including the online items, paper/pencil test materials, and artifacts produced as a result of test administration, are secure. To maintain the validity of the tests administered in the statewide assessment system, security of the test questions and test materials is absolutely necessary. When security is breached, the tests (individually or as a group) are no longer valid—one student, school, or complex area may have accrued advantages not awarded to another, the test is no longer standardized, and is no longer appropriate for program accountability.
Test materials should not be photographed, printed, or reproduced in any way. No digital, electronic, or manual device may be used to record or retain test items, reading passages, or writing prompts. Similarly, these materials must not be discussed with or released to anyone via any media, including fax, email, social media websites, etc. Students who require access to medical monitoring devices during testing should be tested in a separate setting. Descriptions of test items, stimuli, printed reading passages, response options, or printed manipulatives must not be retained, discussed, or released to anyone.
Massachusetts’s AA-AAAS policies also indicated that test security must be maintained and documented at all times for the MCAS-Alt:
MCAS-Alt Administrative and Security Requirements. Principals are responsible for ensuring that all educators administering the MCAS-Alt comply with the requirements and instructions contained in the 2022 Educator’s Manual for MCAS-Alt. In addition, other administrators, educators, and staff within the school and district are responsible for complying with the same requirements. School staff members who violate the test security requirements are subject to the sanctions and penalties outlined in this section. The purpose of the MCAS-Alt security requirements is to protect the validity of the statewide results.
Educators who conduct the MCAS-Alt are responsible for ensuring that information is complete and accurate for each student participating in MCAS-Alt and is properly recorded and included in each student’s binder, as well as on all MCAS-Alt forms and materials, including the Student Information Booklet (SIB). The student’s teacher is also responsible for ensuring that student work samples and other evidence are neither duplicated, altered, nor fabricated in a way that provides information that is false or portrays the student’s performance inaccurately. Evidence for each student, regardless of the similarity of classroom instruction or participation in similar classroom activities, must reflect the individual student’s authentic abilities and performance. The student’s teacher is responsible for the timely submission of student assessments with all required forms and information to their principal for review and sign-off on the Principal’s Certification of Proper MCAS-Alt Administration (PCPA) prior to the submission of binders to the Department.
The DLM consortium specified security practices for states to follow when using its test; many DLM states (but not all) include this information in their policies and post it on their websites. For example, some of the information found on West Virginia’s website said:
DLM alternate assessments are secure assessments.
Braille: When the assessment is complete, the embossed paper form must be securely destroyed. Consult the assessment coordinator about the process to use.
Note that West Virginia also addressed the storage and handling of its AA-AAAS materials in addition to the overall security of the assessment.
Storage and handling of alternate assessment and related materials. As seen in Figure 3, 36 states had policies that addressed the storage and handling of the AA-AAAS and related testing resources. An example of a state policy on security for the storage and handling of its alternate assessment is evident in the policies of Indiana:
It is the responsibility of school officials and CTCs to adhere to all guidelines for the proper disposal and prompt return of secure materials following an assessment administration.
After any administration, initial or make-up, secure materials (e.g., test booklets and TA scripts) were required to be returned immediately to the STC and placed in locked storage.
Georgia indicated requirements for the secure storage of its AA-AAAS testing materials:
Given the unique features of the GAA (such as the test window), test security must be considered and attended to throughout the school year and not just during the portfolio submission phase. Once compiled into the portfolio, student work and materials being used for the purposes of the GAA must be kept in locked storage in the classroom—except during use. Access to those materials must be restricted to authorized individuals only. It is the direct responsibility of all individuals who administer the assessment to follow security procedures and protect the integrity of the assessment process.
Access to assessment and materials. The test security policies of 25 states indicated who could, or could not, access the AA-AAAS and related materials. For example, Vermont’s policy said:
Only students who are testing can view test items. Students who are not being tested or unauthorized staff or other adults must not be in the room where a test is being administered. Trained ATEs/ATAs may have exposure to test items in the course of properly administering the assessments; however, DAs, ADAs, SCs, TEs, TAs,* and other trained staff may not actively review or analyze any test items.
Kentucky had a policy for the security of its Transition Attainment Record (TAR) alternate assessment materials:
When not being used for testing sessions, all AT’s and materials shall be stored in a secure location with access granted to authorized personnel only.
When not being used for testing sessions, all TAR materials shall be stored in a secure location with access granted to authorized personnel only.
Oregon’s policies on access to assessment materials and how certain accommodations can be delivered stated:
Other: Accommodations – Human-based Read Aloud: TAs are allowed to read the text, item prompts, and answer choices in all content areas when administering alternate assessments. The only exceptions are reading items that address standards involving decoding or word identification, or items where independent reading is required, which are not to be read aloud. Standardized test administration protocols will identify these reading items and need to be followed for all items (with appropriate test security). When providing read-aloud support to a student, other interactions between a TA and a student regarding test questions or content are not allowable and may be treated as a testing impropriety.
Other: Accommodation – Answer Choices on Notecard: Notecards must be created on the spot, as a Test Administrator is testing a student. The notecards must be securely destroyed at the conclusion of the testing session. A very small percentage of students may need this accommodation. Educators should follow security procedures when administering this feature.
Speech to Text: If students use their own assistive technology devices, all assessment content should be deleted from these devices after the test for security purposes.
Other. A total of 23 states had test security policies for the AA-AAAS that included other criteria or situations the state wanted to address, such as having others present during testing, conducting online assessments securely, or possible use of electronic devices. For example, Wyoming’s security policy included guidance to others who might be involved in the testing process, such as support staff or translators. It stated that:
Other (Aides, Nurses, Support Staff): Can aides, nurses, or other support staff be present during the administration if they are there for the well-being and support of the student? Must they be trained? Yes. Support staff may be present if the BC has approved it. They are expected to honor test security and sign a Test Security Agreement. They do not need to be trained because they are not administering the test.
Other (Interpreters, translators): Can interpreters or translators be present during the administration for interpreting/ translating questions and responses, even though the interpreter/translator has not attended training? Yes. An interpreter or translator may be present and interpret/translate, even if they have not been trained on the administration of the WY-ALT. They are expected to honor test security and sign a Test Security Agreement. If the interpreter/translator is also the ALT-TA, they must meet all the requirements for serving as an ALT-TA.
In Indiana, which conducts all its assessments online via its vendor’s platform (Cambium), the following policy is in place to ensure that appropriate staff are trained and certified on the security and use of the system and on getting access to it:
TAs must complete a brief certification process annually to initiate assessments in Cambium Assessments, Inc.’s (CAI’s) platform. A separate annual certification is required to administer I AM based on specific protocols used for this assessment.
For Hawaii, several additional security policy practices on test accommodations, who can serve as a TA, and the use of various devices were specified for their AA-AAAS, including:
The use of any accommodation/assistive device that is not a regular part of daily instruction is not allowed; e.g., the student uses an accommodation during testing, but does not use the same accommodation as a regular part of the instructional day.
The HSA-Alt must be administered by a certified Test Administrator, who will most likely be the student’s classroom teacher. Classroom aides or paraprofessionals may not administer the HSA-Alt.
Students are actively supervised by a certified test administrator and are prohibited from access to unauthorized electronic devices that allow availability to outside information, communication among students or with other individuals outside the test environment, or photographing or copying test content. This includes any device with cellular, messaging, or wireless capabilities, but is not limited to cell phones, smart watches, personal digital assistants (PDAs), iPods, cameras, smart watches, and electronic translation devices.
Test Security Policies for Accommodations
Our analysis of 2020-21 policies indicated that many documents and resources were available on the test security policies for a wide variety of accommodations. Some state materials described potential security issues when accommodations were not administered appropriately. Some states provided detailed information about how to securely provide accommodations, while other states gave basic information about the accommodation to reinforce what was in the state’s accommodations or accessibility manual.
Figure 4 indicates the number of states that included each of several selected accommodations in its test security policies. The use of a human reader was mentioned in the policies of 40 states. The only other accommodation included in the test security polices of more than half of the states was transcription and scribing (36 states).
Figure 4. Selected Accommodations Addressed in States’ Accommodations Policies (N=51)
Each of the 15 accommodations shown in Figure 4 is addressed here. They are grouped according to the needs that they address for students with disabilities. Although some accommodations in a group might be considered substitutes for one another, they may have different security risks. In other cases, one accommodation in a group may be used for paper-based tests (PBTs) while the other is used with computer-based tests (CBTs).
For this analysis, the eight groupings of accommodations are:
- Oral delivery accommodations (human reader, text to speech, audio taped administration)
- Accommodations used to record student responses (transcription and scribing, speech to text)
- Enlarged text accommodations (large print, magnification)
- Braille and sign language interpretation accommodations (braille and sign language)
- Scheduling/timing accommodations (extended time, multiple days)
- Setting accommodations (small group administration, individual administration)
- Calculator
- Other
For more specifics and additional details about the ways in which state test security policies addressed accommodations for both the general assessment and the AA-AAAS, see Tables B7a, B7b, and B8 in Appendix B.
Oral delivery accommodations. As shown in Figure 5, there were three types of oral delivery accommodations included in states’ test security policies: human reader, text to speech, and audio-taped administration. The human reader accommodation and audio-taped administration—as well as text-to-speech assistive technology software—are accommodations that are typically used for PBTs. Text to speech (which may be either embedded in the test platform or a separate software application) is more often used for CBTs. For details see Tables B7a and B8 in Appendix B.
Human reader. The test security policies of 40 states addressed the oral delivery of the assessment by human readers, often called a read-aloud accommodation. Many state policies emphasized that the tests contained secure content and that human readers must not disclose test content or read the assessment in ways that might affect the students’ responses. In addition, many states required human readers to sign a test security confidentiality or non-disclosure agreement. For example, the California policy said:
Test readers must be trained in accordance with Smarter Balanced and member administration, as well as security policies and procedures as articulated in Smarter Balanced and Consortium member test administration manuals, guidelines, and related documentation. Test readers should read and sign a test security/confidentiality agreement prior to test administration.
When a non-embedded designated support or accommodation is used that involves a human having access to items (e.g., reader, scribe), procedures must be in place to ensure that the individual understands and has agreed to security and confidentiality requirements.
Figure 5. Oral Delivery Accommodations (N=51)
Some states’ policies emphasized the importance of following guidelines about the appropriate way to read content. For example, the Florida policy stated:
For students with the oral presentation accommodation taking FSA Mathematics, FSA EOCs, NGSSS Science, or NGSSS EOCs, all directions, passages, test questions, and answer choices may be read aloud. Charts, tables, illustrations, and graphs may also be read aloud to students with the oral presentation accommodation. The test administrator or proctor may describe the charts, tables, illustrations, graphs, etc., in a manner similar to that which the student would normally encounter in the classroom, but the test administrator or proctor must exercise care not to use inflection that might lead a student to the correct/ incorrect response.
Text-to-speech. In 2020-21 policies, 18 states addressed the secure administration of the text-to-speech accommodation. When a text-to-speech CBT platform was used, many states required test administrators to have reviewed the material to ensure that reading passages would not be read aloud, all content is pronounced appropriately, and test items were not read in a manner to give away an answer (similar to read aloud). A few states’ test security policies indicated how the text-to-speech accommodation should be provided. For example, New York had a test security policy for specially approved use of this accommodation. It stated:
State assessments provided through computer-based testing have text-to-speech capabilities embedded in the testing platform. If a student’s IEP/504 plan specifies “text-to-speech” as a testing accommodation (or an implementation specification for the “tests read” accommodation), the testing coordinator for that school must be contacted to activate this feature for that student. In situations where a paper copy of a State assessment must be scanned onto a school-sanctioned device, in order to be provided this accommodation in accordance with specifications in an IEP/504 plan, this accommodation would be considered “revised format.” For State assessments, any reproduction and/or reformatting of test booklets requires the advance written permission of the Office of State Assessment. A request to open the test earlier to make these changes needs to be submitted to the Office of State Assessment. The request must be submitted by the principal and must indicate that, based on the student’s IEP/504 plan, permission is needed to revise the format.
New Jersey had this policy on use of a text-to-speech accommodation:
For ELA, text-to-speech is an accommodation. Students must have a valid IEP or 504 plan to access this feature. For math and science, text-to-speech is an accessibility feature. IEPs and 504 Plans are not necessary to use this feature for math and science; however, it should not be administered to all students simply because it is available.
Audio-taped administration. In the past, use of an audio-taped test administration was another way that assessments were delivered orally, either by cassette tape or CD format. Many states used this approach in previous years. However, our analysis found no states addressing this accommodation in its 2020-21 test security policies. In other words, there was no evidence of any state policies about the use of audio-taped administrations. This change in policies probably occurred because most states had moved to online testing by 2021; computers typically have this accommodation or accessibility feature built into their testing platforms.
Accommodations used to record student responses. Some students need an accommodation to record their responses; this type of accommodation was commonplace in state assessments. As shown in Figure 6, there were two types of accommodations for recording student responses (i.e., transcription and scribing, speech-to-text) included in states’ test security policies. The use of a human access assistant to do transcription and scribing was used most often with paper-based tests (PBTs), whereas speech-to-text (which may be either embedded in the test platform or a separate software application) was more often used for CBTs. However, speech-to-text software also was sometimes used with PBTs.
In some cases, a human access assistant transcribes the student’s responses into the standard answer documents. Students who participate in CBTs who are unable to use speech-to-text software (for example, a student with both a physical disability and a speech and language disability) may need a human scribe to enter responses. For details see Tables B7a and B8 in Appendix B.
Figure 6. Accommodations Used to Record Student Responses (N=51)
Transcription and scribing. Our analysis of 2020-21 policies indicated that a majority of states (36 states) had security policies for this accommodation. Many of these policies included guidelines for a human access assistant who provided this accommodation. For example, Vermont’s policy, which is based on Smarter Balanced procedures, said:
The scribe must be trained and qualified and must follow the administration guidelines provided in the Smarter Balanced Test Administration Manual. The use of this support may result in the student needing additional overall time to complete the assessment.
Some policies provided detailed descriptions of exactly how scribing should be provided. Virginia’s policy stated:
The student will dictate in English (or use an augmentative communication device with auditory output) his/her response to the prompt for the short-paper component of the Writing assessment to a school official (scribe), who will transcribe it. The scribe, who should have experience working with the student, must format, capitalize, and punctuate only as directed by the student. The student is not required to spell each word to the scribe. Care must be taken by the scribe not to provide help on test items. Examples of prohibited help include, but are not limited to: discussing test items, providing hints or clues, giving reminders, giving verbal indications or nonverbal cues about the correctness of a student’s answer.
Other state policies were even more specific on the procedures that must be followed to securely transcribe student responses. Washington’s policy stated that:
Scribes should read and sign a test security/confidentiality agreement prior to test administration. Scribes are expected to familiarize themselves with the test format in advance of the scribing session. The online Training Test or Practice Test is one tool that can help the reader gain this familiarity. Having a working familiarity with the Test Delivery System (TDS) will help facilitate the scribe’s ability to record the student’s answers. Scribes may wish to review the practice test to become familiar with the assessment. Scribes should know about any additional designated supports and/or accommodations the student will need to use during the test. This will ensure there are plans in place for providing all needed designated supports and accommodations. Scribes should also have a strong working knowledge of the embedded and non-embedded universal tools, designated supports, and accommodations available on state assessments. Scribes should review this Scribing Protocol for Washington State Assessments with the student at least one to two days prior to the test event. Scribes should practice the scribing process with the student at least once prior to the scribing session.
Speech to text. In their 2020-21 policies, 20 states addressed the speech-to-text accommodation. Many states’ policies included similar information about the use of speech-to-text with assistive technology devices. For example, Wyoming included this information in its policy:
If students use their own assistive technology devices, all assessment content should be deleted from these devices after the test for security purposes.
Colorado’s test security policy provided a lengthy list of requirements for speech-to-text software:
Available to students as documented on their IEP or 504 plans. Because this accommodation requires that the student provide answers orally, this accommodation must be provided in an individual, one-on-one, testing environment. Additional considerations for test security must be applied when students are using speech-to-text (STT) software. Students must not retrieve or access work from another student. Students must not be able to access additional programs/applications or the Internet while testing. Student must not be able to access any previously saved data while in the testing environment. Extra time may not be given for “lost” work. Assistive technology devices with student answers on them are secure test materials and must be secured as such. For students using assistive technology and/or augmentative communication devices that: Do not produce a printed product, perform transcription of responses directly from the device. Do produce a printed product, print student work and transcribe responses into TestNav or the student’s test book prior to the student beginning the next test unit. Verbatim transcription should occur in the presence of a second school adult and follow the transcription guidelines. Student work and/or answers that are not transcribed are not scored. STT programs that have a kiosk mode that lock out all other applications/programs while testing should be used. STT programs that can still access the internet while in kiosk mode are not approved for use by individual students during testing.
Enlarged text accommodations. As shown in Figure 7, two types of enlarged text accommodations—large print and magnification—were included in states’ test security policies. Large print is typically for PBTs and magnification more often is for CBTs. In their 2020-21 policies, a small number of states addressed their requirements for use of these types of accommodations in their documents and materials. For details see Tables B7a and B8 in Appendix B.
Figure 7. Enlarged Text Accommodations (N=51)
Large print. In states’ 2020-21 test security policies, 13 states specifically addressed the large print accommodation. Some states had policies that described the appropriate way to transfer student responses from the large print documents to scannable scoring sheets or into an online system. For example, the Illinois policy stated:
Responses must be transcribed verbatim by a Test Administrator in a standard student test booklet or answer document, which is included in the Large Print Test Kit. At least two persons must be present during transcription of student responses (one transcriber and one observer confirming accuracy). It is recommended that one of the individuals be a DTC or School Test Coordinator.
Other states addressed the security aspects of accessibility features for CBTs, such as large print. For example, the DC policy stated:
A large print paper-based form of each assessment is available for a student with a visual impairment who is unable to take a computer-based assessment. Responses must be transcribed verbatim by a Test Administrator in a standard student test booklet or answer document, which is included in the Large Print Test Kit. Only transcribed responses will be scored. At least two persons must be present during transcription of student responses (one transcriber and one observer confirming accuracy). It is recommended that one of the individuals be an LEA or School Test Coordinator.
Magnification. The test security policies of only four states described how tests may be magnified or enlarged. For example, the Florida policy indicated that:
Students may use magnification devices (e.g., CCTV/video magnifiers, reading loupes, handheld magnifiers). Devices must be used without accessing image-upload features (e.g., devices with a memory card must have the memory card removed during testing).
Oregon’s policy provided information about the provision of the magnification accommodation and maintaining security:
A student may use any visual magnification device that does not compromise the security of the Statewide Assessment.
Use of projection devices—This designated support is consistent with the existing allowance for visual magnification devices and does not compromise the security of the assessment. A secure room and the technology must be available. Room security ensures that the projection screen is not visible to individuals not taking the assessment.
Braille and sign language interpretation accommodations. As shown in Figure 8, the test security policies of many states contained information about braille and sign language interpretation accommodations. For details see Tables B7a and B8 in Appendix B.
Figure 8. Sign Language Interpretation and Braille Accommodations (N=51)
Sign language interpretation. The test security policies of 24 states addressed sign language interpretation. Many of these policies focused on the qualifications of the interpreter, handling of materials, and the appropriate way to securely sign words to ensure a valid administration of the test. For example, the Utah policy said:
The interpreter should have a valid certification and be in good standing with the Utah Interpreting Program (UIP). Utah Code R933-300-303 outlines all required qualifications. It is preferred that the interpreter is a person familiar with the student and content area, but may not be a relative, guardian or friend of the student. The interpreter must be trained on, understand and agree to their obligation as part of the test administration team. The interpreter must sign the Standard Test Administration and Testing Ethics Policy, found on the USBE website: Ethics Policy and Signature Form. The interpreter should access and adhere to all test administration policies and procedures outlined in the Test Administration Manual (TAM) for the given assessment. Test related materials of any kind (including preparation notes) may not be removed from the secure area within the test site prior to testing. Any testing materials (including preparation materials) must be securely handled and turned in to the test administrator. Interpreters may only interpret items that are able to be read aloud or have the text-to-speech option available as outlined for individual assessments. The interpreter should listen to and interpret only the audio portions.
Some states’ policies reminded interpreters of the requirement to sign a security agreement and what they can and cannot do during testing while providing signing to students. For example, the Alaska policy said:
Signing to student. Interpreters must read and sign a Test Security Agreement and may not provide additional information to student, such as drawing pictures of math problems.
Connecticut’s policy specified how this accommodation can be used securely with the state’s Smarter Balanced tests:
The Human Signer for ELA Reading Passages is a sign language accommodation for the Smarter Balanced ELA Reading Passages allowing a qualified test administrator to sign or provide visual language support for the reading passages to a student who is deaf or hard of hearing with a print disability. Students are tested individually and, if necessary, a sign language interpreter may assist in test administration by providing directions and clarifying information as allowed in the Smarter Balanced Test Examiner’s Manual.
Braille. Nineteen states had test security policies in 2020-21 that addressed a range of issues in the provision of the braille accommodation. States described a variety of formats and technology aids that can be used to deliver braille, and the process for securely recording student’s responses. For example, the Mississippi policy stated that:
Students who regularly use braille materials in the classroom qualify for braille materials on selected state assessments. Two certified individuals must transcribe the answers verbatim to a regular test booklet containing the student’s demographic information.
Some states addressed the security of braille materials and training of TAs in their resources, along with test security policies that described the appropriate process for the use of a braille accommodation. For example, Oregon’s policy said:
All embossed braille materials must be collected and securely destroyed at the end of each assessment session to maintain test security.
Prior to administering the assessment through the Braille Interface, test administrators must receive both the general test administration and security training provided locally through the school district, as well as specific training on administering the Science assessment through the Braille Interface and its supporting Braille technologies.
Virginia’s policies provided detailed information on how to administer braille assessments. For example, its policy said:
Examiners/Proctors responsible for monitoring students using braille tests should be familiar with the Braille Notes and Transcriber’s Notes included in the Examiner’s copy of the braille test. The Braille Notes list the differences between the braille test form and the corresponding Examiner’s copy and are intended to help the Examiner answer any questions that a student may have regarding reading and interpreting items in the braille form. Transcriber’s Notes provide information to the (braille) reader that is not readily apparent by reading the test item’s text alone. If a student has a question pertaining to the Transcriber’s Notes on his/her test, the Examiner can consult the Examiner’s copy for assistance.
Blind and vision impaired students may use a braille writer as scratch paper to take notes or complete calculations, to respond to the writing prompt of the short-paper component of the Writing test, or to record responses to multiple-choice questions on the tests. The braille writing device must not be connected to the Internet or the device’s Internet access must be disabled during the test session. Some braille writing devices have features or capabilities beyond those which allow the student to produce written material in braille. Those additional features must be disabled during the test session. If the student requires the use of one of those features, and that feature is not equivalent to a tool that is allowed for all students (e.g. scratch paper), or does not meet the conditions of a specific accommodation provided in this section (e.g. graphic organizer), a Special Assessment Accommodation Request must be submitted to VDOE.
Scheduling/timing accommodations. Figure 9 shows the number of states’ policies that addressed two scheduling/timing accommodations (i.e., extended time, multiple days). For details see Tables B7b and B8 in Appendix B.
Figure 9. Scheduling/timing Accommodations (N=51)
Extended Time. In 2020-21 policies, only seven states addressed security issues related to the provision of an extended time accommodation. This may be due, in part, to the fact that most states provided untimed assessments to all students. However, some special situations requiring extra security were noted in some states’ policies. For example, the Illinois policy said:
Students with extended time accommodations must be given a unit in a continuous block of time and may not be brought back to that unit at a later time. If the accommodation extends into the student’s schedule lunch, then the Test Administrator must either accompany the student to lunch and remain with him or her, or bring the student’s lunch to the testing room.
Washington’s policy provided a description of how extended time testing must be monitored by a test administrator:
When students need extended testing time, they must be monitored by a trained TA. If a student needs to move to a new location with a different TA, the student exits the test session they are currently working in and join the test session of the new TA.
Multiple Days. Six states had policies that addressed security issues related to testing across multiple days. For example, Idaho’s policy stated:
If the test proctor intends to administer the test over the course of multiple days for a student or group of students, test proctors may ask students to pause after they reach a designated point. There is nothing built into the system to prevent students from progressing from one segment of the test to another. The test proctor should give students clear directions on when to pause. For example, test proctors may designate a certain amount of time for testing. Likewise, the end of Part 1 of the ELA/literacy PT might be a logical stopping point. This guidance may be written on a dry-erase board, chalkboard, or another place that students can see easily. Students receive a notification when they reach the end of the segment.
Maryland’s policy addressed the requirement of obtaining permission from MSDE if testing needed to be done over multiple days and how to balance multiple day testing with the need for test security:
In certain limited situations, permission may be granted by the state department of education (MSDE) for a student to have one or more of the assessments administered over one or more days. This type of administration is not typically allowed because of test security issues related to extending the test period beyond the scope of one single day. However, if a student has identified in his or her IEP or 504 Plan the accommodation 5b (Unique Timing and Scheduling Accommodations), the MSDE will consider allowing that accommodation to take precedence over test security considerations.
Setting. Figure 10 shows the results for two setting accommodations—small group administration and individual administration—that were included in states’ test security policies. For details see Tables B7b and B8 in Appendix B.
Figure 10. Setting Accommodations (N=51)
Individual administration. The test security policies of 10 states addressed the use of this accommodation. Many of these policies provided little detail about how individual administration should be carried out except for saying a separate secure location away from other students was needed. For example, the Louisiana policy said:
Students who have the accommodation Individual Administration must be tested one-on-one by a test administrator in a location separate from other students. With the support of the school administration, the school test coordinator also has the authority to schedule students in testing spaces other than regular classrooms, and at different scheduled times, as long as all requirements for testing conditions and test security are met as set forth in this manual.
DC’s policy addressed how this accommodation could be used with other accommodations, such as read aloud, human reader, or scribe. It stated:
Students receiving accommodations that require student speaking (e.g., student reads aloud to themselves) must be tested in a one-on-one setting. Students receiving a human reader accommodation must be tested in a one-on-one setting or a small group setting with only students receiving the human reader accommodation. Students receiving the human scribe accommodation must be tested in a one-on-one setting.
Small group administration. The policies of six states discussed the accommodation of small group administration. Many of these policies provided little detail about how small group administration should be carried out. This accommodation often was mentioned in conjunction with individual administration and also often within the context of the provision of another accommodation (e.g., read aloud, scribe, sign interpretation, etc.). For example, the Louisiana policy said:
Tests may be administered to an individual or a small group of students (maximum of 8) who require more attention than can be provided in a larger classroom. If a student has other accommodations that affect the standard administration of the test (Answers Recorded), individual or small group administration must be used. The school test coordinator should plan for this while planning for all logistics of testing. Students testing in small groups must be seated an adequate distance apart to prevent copying. With the support of the school administration, the school test coordinator also has the authority to schedule students in testing spaces other than regular classrooms, and at different scheduled times, as long as all requirements for testing conditions and test security are met as set forth in this manual.
The policy in Hawaii addressed the use of both small group and individual administration as accommodations that can benefit some students; it provided additional details on how these administrations can be conducted in a secure manner:
Students who are easily distracted (or may distract others) in the presence of other students, for example, may need an alternate location to be able to take the assessment. The separate setting may be in a different room that allows them to work individually or among a smaller group. The student may read aloud to self, use a device requiring voicing (e.g., a Whisper Phone), or use Amplification. It may also include a calming device or support as recommended by educators and/or specialists. Or, the separate setting may be in the same room but in a specific location (for example, away from windows, doors, or pencil sharpeners, in a study carrel, near the teacher’s desk, or in the front of a classroom). Some students may benefit from being in an environment that allows for movement, such as being able to walk around. In some instances, students may need to interact with instructional or test content outside of school, such as in a hospital or their home.
Calculator. As shown in Figure 11 the test security policies of 20 states addressed the calculator accommodation. Their policies often described the types of calculators that were allowed. Further, with most state assessments in 2020-21 being given online, the calculator accommodation usually was embedded in the computer platform or secure web browser. Thus, the types of calculators have mostly been standardized and made secure as part of the online system. Still, some students require non-embedded calculators, and those were addressed in policies. For example, the Florida policy indicated:
Handheld scientific calculators must be on the approved list or have only the allowable functionality indicated in the Calculator and Reference Sheet Policies for Florida Statewide Assessments document (available on the portal). If providing handheld calculators, ensure that students are provided handheld calculators in the appropriate test sessions only. Providing a calculator with prohibited functionality (e.g., a display of more than one line, graphing capability) or in the wrong test session (Session 1) is cause for test invalidation.
For details see Tables B7b and B8 in Appendix B.
Figure 11. Calculator Accommodation (N=51)
The Delaware policy addressed the possibility that stand-alone non-embedded calculators may still be needed as an accommodation:
Test administrators should ensure that the calculator is available only for designated calculator items and that calculator functions are consistent with those of the embedded calculator for each grade level. The non-embedded calculator should have no internet or wireless connectivity, and all security procedures need to be followed.
Other. Nineteen states had test security policies that addressed the use of other types of accommodations. This group covers a wide variety of accommodations that may be offered during state assessments. Figure 12 shows the number of states addressing test security for “Other” types of accommodations.
Figure 12. “Other” Accommodations (N=51)
The types of “other” accommodations we identified in our analyses (and the states with them) included:
- Administering tests in non-school settings (AL)
- Use of paper versions of CBTs (AZ)
- Prohibition of certain accessibility supports (AR)
- Use of other types of approved accommodations (LA)
- Unique accommodation requests (MA)
- Mode options (MI)
- Use of assistive technology (MS)
- Use of spell checking devices, manipulative devices, math spatial boards (NY)
- One item per page format, AT devices (NC)
- Oral translations, use of music and white noise, dual monitors (OH)
- Print on demand, musical supports, word prediction (OR)
- Familiar test administration (RI)
- Photocopying (TX)
- Amplification, printouts (WI)
- Audio devices, scratch paper, word prediction (WY)
For additional details and more specifics on the state test security requirements for accommodations in the “Other” category, see Table B7b and B8 in Appendix B.
Discussion
Most states had many more documents and materials (e.g., state technical reports, test administration manuals, test coordinator manuals, accessibility and accommodations manuals, training materials) that addressed test security for their assessments in 2020-21 than they had in 2013-14. Policies in 2020-21 were more likely to include detailed information about how to appropriately assess students with disabilities than in 2013-14.
Major Findings
This analysis of 2020-21 policies found that there were many more documents and materials than in 2013-14 from states on their test security policies, including those that included information about students with disabilities, test accommodations, and alternate assessments (AA-AAAS).
The topics most often addressed for students with disabilities in states’ 2020-21 test security policies were:
- Training requirements related to accommodations
- Role of IEP
- Security of tests and materials for accommodated tests
- Duplication and copying of test materials
- Qualifications of test administrators and accommodations providers
- Changes to standard test procedures
- Personnel knowledgeable about special education requirements
- General information about accommodations security
Overall, the three most frequently cited topics were security of tests and materials for accommodated tests (44 states), role of IEP (43 states), and training requirements related to accommodations (41 states). Many states had test security manuals (17 states) that included information about students with disabilities and the assessment they take (15 states). Many also had confidentiality agreements that included requirements related to accommodations or alternate assessments that test coordinators, test administrators, proctors, and accommodations providers must sign (22 states). They also had procedures for documenting irregularities and incidents related to the non-provision or mis-provision of accommodations on a student’s IEP (41 states), as well as monitoring procedures that were inclusive of alternate assessments and monitoring for the provision of accommodations (38 states).
New Themes. Among several new themes were the findings that some states had developed test security manuals that brought together the state’s policies, practices, and procedures related to test security. These manuals often included information about students with disabilities and the tests they take. They served as a single stand-alone source of information on test security in a variety of state materials. In many states, information about test security for students with disabilities was presented in several separate documents. Sometimes information was different in different documents, a situation that causes confusion and in turn has an effect on test security. The development of a Test Security Manual that brings all information together into one comprehensive document is a valuable improvement because it helps make things more consistent.
Another new finding from the analysis of 2020-21 policies was the expanded use by states of signed confidentiality agreements or affidavit forms for maintaining test security. This approach makes it clear to those involved in administering assessments that security is to be taken seriously and that there may be legal implications if procedures are not followed.
An additional new finding about 2020-21 policies was the development of more detailed procedures and plans for monitoring assessments for the provision of accommodations and alternate assessments. These are practices required by the U.S Department of Education peer review guidance, so it is not surprising that states had developed more comprehensive policies on ensuring that the security of their assessments was given adequate attention.
Security for Accommodations. States had a wide variety of security policies for test accommodations in 2020-21. Among the accommodations most frequently cited in state documents were the use of a human reader (40 states), transcription and scribing (36 states), sign language interpretation (24 states), calculator (20 states), and speech-to-text (20 states).
Accommodated assessments that involve a human to administer accommodations are among the assessment administrations that are more likely to have test security breaches. These accommodated administrations often occur in a small group or individual setting, which may be monitored less often than other settings and may have an individual administering the assessment who is less qualified or has received less training than other assessment administrators. State test security policies need to address specifically how these test administrations should occur to minimize the likelihood of a security breech. States should consider whether their test security policies need to provide more guidance about small group and individual administrations.
Security for Online Assessments. In the years since the analysis of 2013-14 test security policies (Lazarus et al., 2014), there has been a huge shift to CBTs. As states have moved to online assessments, the platforms they use now include many embedded accessibility tools and accommodations that are provided via technology. As general assessments and alternate assessments moved online, states also had to implement procedures for their secure administration. For example, many states introduced security policies for online assessments that addressed the speech-to-text and text-to-speech accommodations. These policies often state that care needs to be taken to help ensure that the technology is not providing the student with access to any outside information or an accommodation not allowed for that student.
Accommodations that used to be provided by human access assistants are now done online (e.g., text-to-speech instead of human reader; or speech-to-text instead of scribe). Although a technology-embedded approach may reduce some security risks, it is likely that there will continue to be a need for human access assistants (for example, for a student with physical disabilities who needs a human scribe because of disability-related motor skill or speech issues). Thus, continued attention should be given in states’ test security policies to the security of accommodations provided by humans.
Security for AA-AAAS. The type of AA-AAAS (e.g., items, performance tasks) that states administered varied across states. These format differences were sometimes reflected in the way state policies approached test security. In general, test security policies were more detailed and specific in 2020-21 policies than in 2013-14 policies. Most often, the policies in 2020-21 addressed the security of the assessment and AA-AAAS materials and the storage and handling of the AA-AAAS.
Considerations
Considerations for Accommodations. Where there are several accommodations that serve similar purposes (e.g., human reader/audio-taped recording/text-to-speech, scribe/speech-to-text, extended time/multiple days, magnification/copies of test materials enlarged by duplicating), states may want to consider identifying the accommodation with the fewer risks as the preferred accommodation. For example, text-to-speech would generally be preferred over either human reader or audio-taped recording. Of course, the decision for an individual student with a disability would depend on that student’s needs.
Considerations for Alternate Assessments. Although many of the listed considerations focus on students with disabilities and test accommodations for the general assessment, states also need to attend to test security for their AA-AAAS. This analysis found that 35-40 states addressed the AA-AAAS in their test security documents. This means that 10-15 states did not. AA-AAAS may require special attention and additional procedures to maintain security for students with the most significant cognitive disabilities who take these tests (and the proctors that administer them). Because many of these students require one-on-one administration, it is important for all states to have strong policies in place for the security of their AA-AAAS.
Considerations for Remote Testing. Remote testing for students with disabilities needs to be addressed in test security policies. For many students with disabilities taking a general assessment, remote testing may work, with the same security procedures as for their peers (Lazarus et al., 2021). For students with disabilities who need a human-provided accommodation or an AA-AAAS, the question of whether it can be done securely when administered remotely is a question. States should consider whether it is appropriate for parents to administer accommodated assessments at home. One possible option is to bring students to school or another safe and secure site for the assessments. Whether this is possible should be evaluated to determine whether this practice may cause other unintended consequences.
Questions to Consider When Including Students with Disabilities in Test Security Policies. Several questions should be considered when making decisions on the secure testing of students with disabilities. Specific state policies may be needed to address and document them appropriately. Questions include:
- Are all test administrators, proctors, and accommodations providers qualified?
- Does everyone who accesses the assessment, including those preparing for and administering accommodated assessments and alternate assessments, sign a test security affirmation form or confidentiality agreement?
- Are adequate records kept on training, monitoring, irregularities, etc.?
- Are accessibility features and accommodations with the fewest security risks identified as preferred accessibility features and accommodations?
- Are test security procedures in place for situations when a test is accessed prior to administration (e.g., to prepare to sign an assessment, to make adjustments to test materials, etc.)?
- Are appropriate procedures used for small group and individual administrations?
- Are appropriate test security procedures in place for AA-AAAS?
- Are appropriate test security procedures in place for the administration of accommodated tests and AA-AAAS during remote testing?
Limitations of the Analysis of 2020-21 Policies
For this study, many state documents that addressed test security and students with disabilities were compiled and analyzed, but it is possible there were additional state documents with information about test security policies for accommodated tests for students with disabilities, alternate assessments, and other related issues. This analysis included only documents that were publicly available on states’ websites. Documents available on secure portals that required a user log-in, as well as hard copies sent to districts and schools, were not analyzed.
Only data from state policies that pertained to students with disabilities and test security or confidentiality were compiled, coded, and analyzed. Therefore, other information about test security (e.g., how tests were handled and stored, general assessment procedures, etc.) was not included when it did not specifically mention students with disabilities or accommodations.
Suggestions for Improving Test Security Policies and Procedures
The ability to make valid inferences is compromised when assessment procedures are not followed and when accommodations are used inappropriately. Policies, processes, and procedures are needed that will support the valid measurement of what students with disabilities know and can do while maintaining test security. States should thoughtfully consider how to balance test security and accessibility. Suggestions include:
- If a state does not have a test security manual, it should consider developing one to help ensure that consistent, current information is readily available. The manual should include information about the participation of students with disabilities, test accommodations, and AA-AAAS.
- States should require that all test administrators, proctors, and accommodations providers sign an affirmation form or confidentiality agreement prior to administering any tests. These agreements should specifically address issues related to the assessment of students with disabilities (e.g., accommodations, AA-AAAS).
- States should have plans in place for following up on testing irregularities and incidents, with specific information on testing irregularities related to the provision (or non-provision) of accommodations.
- States should ensure all test administrators and accommodations providers receive training that specifically addresses how to maintain test security while providing access for students with disabilities.
- States should have policies in place for situations where test administrators or accommodations providers may need to access the test prior to administration (e.g., to prepare for sign interpretation, to make adjustments to test materials, etc.). This will help ensure that security is addressed in these situations.
- States should annually review their test security policies and procedures to ensure that they adequately address security issues related to the use of any embedded access tools and accommodations that are provided on computer-based tests.
References
American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education Joint Committee on Standards for Educational and Psychological Testing. (2014). Standards for educational and psychological testing. AERA. http://www.apa.org/science/programs/testing/standards.aspx
Council of Chief State School Officers and Association of Test
Publishers. (2013). Operational best practices for statewide large-scale
assessment programs: 2013 Edition.
http://www.ccsso.org/resources/publications/operational_best_practices_for_statewide_large-scale_assessment_programs.html
http://www.amazon.com/Operational-Practices-Statewide-Large-Scale-Assessment/dp/149109642X
Cizek, G. J. (1999). Cheating on tests: How to do it, detect it, and prevent it. Lawrence Erlbaum.
Fremer, J., & Olson, J. (2015). TILSA test security: Lessons learned by state assessment programs in preventing, detecting, and investigating test security irregularities. Council of Chief State School Officers. https://ccsso.org/resource-library/tilsa-test-security
Government Accountability Office. (2013). K-12 education: States’ test security policies and procedures varied (GAO-13-495R). http://www.gao.gov/products/GAO-13-495R.
Lazarus, S. S., & Thurlow, M. L. (2014). Considerations when including students with disabilities in test security policies (Policy Directions 23). National Center on Educational Outcomes. http://education.umn.edu/NCEO/OnlinePubs/Policy23/.
Lazarus, S. S., Thurlow, M. L., Dominguez, L. M., Kincaid, A., & Edwards, L. M. (2014). Test security and students with disabilities: An analysis of states’ 2013-14 test security policies (Synthesis Report 95). National Center on Educational Outcomes. http://www.cehd.umn.edu/NCEO/OnlinePubs/Synthesis95/default.html.
Lazarus, S., Thurlow, M., & Warren, S. (2021) Restart & recovery: Meeting the assessment needs of students with disabilities during times of interrupted schooling. Council of Child State School Officers. https://753a0706.flowpaper.com/CCSSORRAssessmentNeeds/#page=1
Olson, J., & Fremer, J. (2013). TILSA test security guidebook: Preventing, detecting, and investigating test security irregularities. Council of Chief State School Officers. http://www.ccsso.org/Resources/Publications/TILSA_Test_Security_Guidebook.html.
U.S. Department of Education. (2018). Peer review guidance—A state’s guide to the U.S. Department of Education’s assessment peer review process. U.S. Department of Education. https://www2.ed.gov/admins/lead/account/saa.html#Standards_and_Assessments_Peer_Review
Wollack, J. A., & Fremer, J. J. (Eds.). (2013). Handbook of test security. Routledge.