NCEO Policy Directions

Published by the National Center on Educational Outcomes
Number 16 / October 2003


Putting it All Together: Including Students with Disabilities in Assessment and Accountability Systems

Prepared by Cammy Lehr and Martha Thurlow


Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Lehr, C., & Thurlow, M. (2003). Putting it all together: Including students with disabilities in assessment and accountability systems (Policy Directions No. 16). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/Policy16.htm


Background

For the past decade, states across the nation have been setting high academic content standards for all children, and developing assessment and accountability systems to ensure that all children learn to very high levels. Federal legislation, including the Individuals with Disabilities Education Act Amendments of 1997 (IDEA) and the No Child Left Behind Act of 2001 (NCLB), provides guidelines and expectations for implementation of inclusive instruction and assessment. States, districts, and schools are now keenly attuned to state assessments that are used to measure student achievement on academic content standards.

Essential components of inclusive assessment systems that must be understood and addressed are student participation in assessments, testing accommodations, alternate assessments, reporting results, and accountability. The implementation of these components directly influences the extent to which inclusive policies and practices become reality. The purpose of this issue of Policy Directions is to provide an overview of the key components of inclusive assessment and accountability and to highlight how they fit together to form a cohesive whole that facilitates the intended benefits of standards-based reform. Although the components are interrelated, each is considered separately to address issues and opportunities.


Assessment Participation

Students with disabilities are required by Federal legislation to participate in state assessments, in part because assessments are key components of educational accountability. They are used to provide information on the educational progress of students and the extent to which students are achieving state standards.

In the early 1990s, students with disabilities often were excluded from participating in state and district assessments. Not only did this lack of participation result in inaccurate pictures of the success of educational programs, but there were also other unintended consequences such as increased referrals to special education, low expectations for students with disabilities, and programmatic decisions based on incomplete or inaccurate information.

Participation in large-scale assessments is now recognized by many educators and parents as a critical element of equal opportunity and access to education. This is true for all students, including students with disabilities and English language learners. Assessments can be used to measure the extent to which schools are including all students in standards-based reforms, monitor the degree to which instructional strategies are helping all students achieve at high levels, and identify curriculum areas that need improvement for specific groups of students.

Students with disabilities can participate in assessments in three ways:

The availability of these three options to all students varies from state to state, but all states provide these options to students with disabilities. Other options are available in some states, including partial participation in testing, more than one alternate assessment, and out-of-level testing. Some of these additional testing options are controversial and require further research to identify risks and benefits. The way that a student participates in statewide assessments must be carefully considered–evaluating intended and unintended consequences is critical.


Assessment Accommodations

Accommodations are changes in testing materials or procedures that enable students to participate in state or district assessments in a way that assesses abilities rather than disabilities. They are provided to "level the playing field." Without accommodations, an assessment may not accurately measure an individual student’s knowledge and skills. The most frequently allowed and frequently used accommodations are shown in Table 1.

Table 1. Accommodations Most Frequently Allowed in Policy and Most Frequently Used in Assessments

Most Frequently Allowed* Most Frequently Used **
Braille Edition Extended Time
Computer Response Read Aloud
Scribe Scribe/Dictation
Extended Time Paraphrasing
Sign Language Interpreter for
Directions
Small Group
Large Print Edition  
Responses Directly Written on Test Booklet  
Read Aloud  
Clarification of Test Directions  
Breaks During Testing  

*Thurlow, M., & Bolt, S. (2001). Empirical support for accommodations most often allowed in state policy (Synthesis Report 41). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.
**Thurlow, M. (2001). Use of accommodations in state assessments: What databases tell us about differential levels of use and how to document the use of accommodations (Technical Report 30). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Providing test accommodations has been shown to increase the participation rates of students with disabilities in assessments. Allowing students to use accommodations is required by law. It is important to remember that an assessment accommodation should be provided because of a student need, not to give a student an unfair advantage. When students with disabilities use assessment accommodations, it is to show what they know without being impeded by their disability.

There is no set of universally approved assessment accommodations, and state policies on reportable accommodations vary tremendously. It is not uncommon to find an accommodation that is "okay" in one state yet "not okay" in another. Some accommodations are believed to change what is being tested, yielding scores that are considered invalid. These kinds of accommodations may be referred to as "nonstandard" or "invalid." Use of nonstandard accommodations may affect what is reported, and states may remove these scores from summary reports, flag them, or report them as zero scores. Policymakers must consider the implications of these policies for accountability formulas and documentation of increased student performance.

Current thinking suggests that when accommodations are used, the degree to which each accommodation is expected to influence test validity should be considered and indicated, perhaps to the level of specific parts of a test. While research on accommodations is growing rapidly, and numerous resources such as NCEO’s online searchable database are now available, it is likely that policy will always play an essential role in accommodation decisions. Thus, it is critical that policy be based on the most up-to-date research and strong theoretical underpinnings.

The Individualized Education Program (IEP) team, including general education teachers and others who know the educational needs of the student, should make decisions about who needs assessment accommodations. Students should play a significant role, with the support of their IEP teams, in the selection and use of assessment accommodations. Necessary accommodations for participation in an assessment must be documented on the IEP.

Decisions about assessment accommodations should be based on what students require to have an equal opportunity to show what they know without impediment of their disabilities. It is important that accommodations do not compromise what the test is measuring. This underscores the importance of making sure decision makers know the purpose of the assessment and the skills or constructs it measures.

Decisions about when accommodations should be used must be carefully considered on an individual basis. Students need to use selected accommodations routinely during classroom instruction to ensure that they are able to use the accommodations with ease on test day. In addition, educators and administrators must be aware of the accommodations that are considered standard or approved for use in a given state. If a nonstandard accommodation is used, it is important to understand the ramifications of its use on a district or statewide assessment (e.g., score may not be counted, denial of diploma).


Alternate Assessments

Alternate assessments are used to evaluate the performance of students who are unable to participate in general state assessments even with accommodations. An alternate assessment provides a mechanism for students with significant cognitive disabilities and other students who may be difficult to assess to be included in the accountability system.

Guidelines for determining whether a student should participate in an alternate assessment typically suggest a very small percentage of students (for example, less than 1% of total). In many states, decisions about participation in alternate assessments have included considerations such as whether students require substantial modifications to academic content standards at grade level, and whether they require intensive individualized instruction to acquire and generalize knowledge.

Statewide alternate assessments were first required as an assessment option in IDEA. NCLB regulations extend the alternate assessment requirements by specifying that each state, district, and school must be held accountable for the achievement of all students, including those participating in the alternate assessment. These assessments are intended to be aligned to academic content standards and to provide the missing piece that makes it possible to include all students with disabilities in state and district assessments and accountability systems. The approaches that states are using for their alternate assessments generally are different from those used for the general assessment. Table 2 summarizes the approaches currently used by states.

Table 2. Alternate Assessment Approaches

Portfolio A collection of student work gathered to demonstrate student performance on specific skills and knowledge, generally linked to state content standards. Portfolio contents are individualized, and may include wide ranging samples of student learning, including but not limited to actual student work, observations recorded by multiple persons on multiple occasions, test results, record reviews, or even video or audio records of student performance.
IEP Linked Body of Evidence A collection of student work demonstrating student achievement on standards-based IEP goals and objectives, measured against predetermined scoring criteria. This approach is similar to portfolio assessment, but may contain more focused or fewer pieces of evidence, with IEP documentation available to support scoring processes. This evidence may meet dual purposes of documentation of IEP progress and the purpose of assessment.
Performance Assessment A direct measure of student skills or knowledge, usually in a one-on-one assessment. These can be highly structured, requiring a teacher or test administrator to give students specific items or tasks, similar to pencil/paper traditional tests, or it can be a more flexible item or task that can be adjusted based on student needs.
Checklist A list of skills, reviewed by persons familiar with a student who observe or recall whether students are able to perform the skills, and to what level.
Traditional (pencil/paper or computer) test A set of traditionally constructed items requiring student responses, typically with a correct and incorrect forced-choice answer format. These can be completed independently by groups of students with teacher supervision, or they can be administered in one-on-one assessments with teacher recording of answers.

From Quenemoen, R.F., Thompson, S.J., & Thurlow, M. (2003) Measuring Academic Achievement of Students with Significant Cognitive Disabilities: Building Understanding of Alternate Assessment Scoring Criteria. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

The focus of alternate assessments has shifted from measuring only functional skills to measuring student achievement of state academic standards. This shift to standards-based alternate assessment measurement approaches has occurred in most states, reinforced by regulations and guidance from Federal policy.

Results from alternate assessments are now included in public reports and incorporated into accountability systems. According to proposed NCLB rules, states may set alternate achievement standards for alternate assessments for students with significant cognitive disabilities but only up to 1% of the total student population can be measured as proficient using these alternate achievement standards. Whether states measure the achievement of students with significant cognitive disabilities against grade-level achievement standards held for all students, or measure against alternate achievement standards that reflect professional judgment of high learning standards for this small percentage of students, this new area of large-scale assessment will continue to require thoughtful policy formulation and implementation.


Reporting Assessment Results

Public reporting of educational results has become an important tool in ensuring accountability for students attaining higher academic standards. Federal mandates emphasize the importance of establishing on-going reporting systems, and require states to disaggregate assessment data for specific groups of students, including students with disabilities. Almost all states now publish at least one statewide educational accountability report on the condition of public education, and some have as many as five or six reports.

Reporting information on students with disabilities is important because it ensures that the performance of these students is visible. In the past, failure to report the assessment results of students with disabilities was a common way to avoid acknowledgment of whether they were benefiting from their educational experiences.

States should consider several principles of best practice when developing policies and practices in reporting assessment data for students with disabilities:

Accountability is a critical aspect of standards-based reform. The rationale for accountability systems is the belief that education can be improved when clear standards for student achievement are communicated to students and educators, achievement toward those standards is measured, and appropriate consequences are linked to levels of student achievement (see Table 3). The intent of these accountability systems is to promote change among schools in ways that will increase positive outcomes for all students.

Table 3. Philosophy of Accountability and Standards-based Reform

“…the idea of standards based reform states that, if states set high standards for student performance, develop assessments that measure students performance against the standards, give schools the flexibility they need to change the curriculum, instruction, and school organization to enable their students to meet the standards and hold schools strictly accountable for meeting performance standards, then student achievement will rise.”
(National Research Council, 1999, p. 15)

This theory of action of standards-based reform is expanded to reflect an education improvement system that includes professional development and improved teaching.


All states are now implementing accountability plans that focus on the system, with consequences assigned to schools, administrators, teachers, and other educators. Some states, but not all, are also using student accountability mechanisms designed to motivate students to do their best. It is now clear that system accountability must apply to everyone in the educational system, including students with disabilities and English language learners. Because schools are likely to target resources toward those students who are included in the determination of rewards and sanctions, a potential consequence of failing to include students with disabilities in accountability systems is that their instructional needs will not be addressed or met.


Putting It All Together

By putting together the essential components of inclusive assessment systems, educational systems can reach a point where each and every student can benefit from standards-based reforms. Each piece of the puzzle is critical and must be addressed to successfully complete the picture and achieve the desired results. The inter-relationships among the components–participation, accommodation, alternate assessment, reporting, and accountability policies–must be considered carefully, along with the intended and unintended consequences of various relationships.


Resources

2003 State Special Education Outcomes: Marching Forward. Thompson, S., & Thurlow, M. (2003). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Alternate Assessments for Students with Disabilities. Thompson, S., Quenemoen, R., Thurlow, M., & Ysseldyke, J. (2000). Thousand Oaks, CA: Corwin Press.

NCEO Online Accommodations Bibliography. Available at http://cehd.umn.edu/NCEO/AccomStudies.htm.

Principles and Characteristics of Inclusive Assessment and Accountability Systems (Synthesis Report 40). Thurlow, M., Quenemoen, R., Thompson, S., & Lehr, C. (2001). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. See http://cehd.umn.edu/NCEO/OnlinePubs/Synthesis40.html

A Self-study Guide to Implementation of Inclusive Assessment and Accountability Systems: A Best Practice Approach. Quenemoen, R.F., Thompson, S.J., Thurlow, M.L., & Lehr, C.A. (2001). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. See http://cehd.umn.edu/NCEO/OnlinePubs/workbook.pdf

Testing Students with Disabilities: Practical Strategies for Complying with District and State Requirements (2nd ed.). Thurlow, M.L., Elliott, J.L., & Ysseldyke, J.E. (2003). Thousand Oaks, CA: Corwin Press.

Testing, Teaching and Learning: A Guide for States and School Districts. National Research Council (1999). Washington, DC: Author.