Are We there Yet? Accountability for the Performance of Students with Disabilities

NCEO Technical Report 33

Published by the National Center on Educational Outcomes

Prepared by:

Sara Bolt, Jane Krentz, and Martha Thurlow

November 2002

Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Bolt., S., Krentz, J., & Thurlow, M. (2002). Are we there yet? Accountability for the performance of students with disabilities (Technical Report 33). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Technical33.htm

Executive Summary

Accountability systems that provide rewards, sanctions, or assistance to schools based on student achievement outcomes are currently being developed and implemented in nearly every state. The purpose of this study was to identify and describe these systems, as well as to determine the degree to which publicly available documents clearly articulate whether students with disabilities are included in accountability calculations. A search was conducted on each state educational agency Web site for information pertaining to school-level accountability. State accountability and Title I directors were also contacted to provide additional information. One of the greatest challenges in studying accountability systems is their transitory nature.

Results of our search illustrate that many systems are currently in a stage of revision. Primary indicators in current school accountability systems include student performance on statewide assessments, performance growth on statewide assessments, attendance rates, and dropout rates. Although it is often clear that students with disabilities participate in statewide assessments, it is frequently less clear whether their scores count in school accountability calculations. In only a handful of states is it entirely clear that all students with disabilities truly count in the determination of consequences for schools.

Overview

Statewide accountability systems designed to promote student achievement are beginning to play a very significant role in the way resources are allocated to schools. The rationale for implementing accountability systems stems from a belief that education can be improved when clear standards for student achievement are communicated to both students and educators, achievement toward those standards is measured, and appropriate consequences are linked to levels of student achievement. The intent of these accountability systems is to promote behavior change among schools in ways that will increase positive outcomes for students.

One of the challenges of researching accountability and assessment systems is that these systems are constantly being revised due to legislative and administrative action. Historically, states and local school boards have determined education and educational accountability policies. However, in the past decade the federal government has played an increasingly prominent role in this area.

The Web sites of major organizations such as Education Commission of the States (http://www.ecs.org), the National Conference of State Legislatures (http://www.ncsl.org) and the Council of Chief State School Officers (http://www.ccsso.org) provide interesting perspectives on the increased attention to assessment and accountability. Many states are attaching high stakes consequences to students for their performance on assessments, such as withholding diplomas (Center on Education Policy, 2002; Guy, Shin, Lee, & Thurlow, 1999; Johnson & Thurlow, 2002) and grade retention (Quenemoen, Lehr, Thurlow, Thompson, & Bolt, 2000), as well as attaching significant consequences to schools and school districts for the overall performance of students (Krentz, Thurlow, & Callender, 2000). The latter approach, known as system accountability, occurs when educators, administrators, schools, or districts are held responsible and consequences assigned for various student performance indicators. For example, school accreditation may be based on test scores; teachers may receive cash rewards for student performance; administrators may be either rewarded or sanctioned based on school improvement, which, in turn, is measured to a large degree by student performance.

As states have proceeded in constructing their student and school or district accountability systems, federal requirements have made it clear that students with disabilities must be included in accountability. Initial efforts to include students with disabilities in assessment systems have realized significant progress in assessment participation and the development of alternate assessments for those students unable to participate in regular assessments (Thurlow, 2000). In order to promote the positive intended consequences of accountability systems for all students, Linn (2000) points to the importance of including all students in accountability calculations. The next step, therefore, is to identify ways in which to include the scores of students with disabilities in accountability systems.

The concept of including the assessment scores of students with disabilities in accountability is simple on the face of the matter, but there are several complications associated with including the scores obtained by students with disabilities that challenge accountability systems. These complications include what have been called "non-standard" administration of assessments (i.e., assessments in which students have used accommodations that are considered to change what the test is intended to measure), and alternate assessments, which generally are designed for students with significant cognitive disabilities, and which therefore seem to be "different" from regular assessments.

Despite these complications, it is important to include students with disabilities in accountability systems, particularly those with school or district consequences. Therefore, it is also important to examine states’ progress toward this end. The purpose of the study reported here was to do just that. We focused on statewide system accountability with rewards, sanctions, or assistance provided for schools or districts. We examined Web sites of state educational agencies to determine whether accountability systems were in place, were being phased in, or were in the development phase. We looked at various indicators used by states to determine rewards or sanctions, how these indictors were determined, whether a growth factor was included, and also noted the various labels applied to schools or districts. We also examined what states are using to determine whether schools receiving Title I services are making adequate yearly progress (AYP).

We then investigated the status of the inclusion of students with disabilities in these systems. With the reauthorization of the Individuals with Disabilities Education Act in 1997 (IDEA 97), states and districts must include students with disabilities in their assessments. They are also required to report on the performance of students with disabilities, both aggregated with other students, and disaggregated. Although it is clear that students with disabilities are required to participate in statewide assessment systems, IDEA 97 did not specifically require that their scores be incorporated into system accountability. Because schools are likely to target resources toward those student groups that are included in the determination of rewards and consequences, a potential consequence of failing to include students with disabilities in accountability systems is that their instructional needs will not be met. Our goal, therefore, was to determine the extent to which students with disabilities are clearly included in accountability systems.

In 2000, we did a similar analysis (Krentz et al., 2000) and found that it was very difficult to locate evidence on state Web sites that clearly stated that students with disabilities were included in accountability systems. Although students with disabilities may have been included in assessments, their scores may not have been included in the accountability systems; most states did not specifically state whether students with disabilities were included when defining the indicators used for accountability purposes.

In 2001, the passage of the Elementary and Secondary Education Act (ESEA), also known as "No Child Left Behind," federally mandated accountability as one of its reform pillars. Annual testing is its cornerstone. While this legislation is in its early stages of being interpreted and implemented, it is apparent that nearly every state will be required to revise its accountability system for all students, including students with disabilities and low-income students receiving services through Title I. According to Wenning, Herdman, and Smith (2002), the required disaggregation of assessment data by various student groups as articulated in No Child Left Behind (NCLB) represents a significant improvement over past practices in which the use of schoolwide averages for accountability purposes frequently masked the performance of various student groups.

As a result of NCLB, many states that have developed accountability systems for schools receiving funds through Title I will now have to broaden their systems to include all students. States may simply include the additional students within their existing accountability systems, or may decide to undergo significant revamping of their current systems in order to comply with ESEA. In several states, the legislature was no longer in session by the time the details of ESEA implementation were shared with state departments of education. Many states have not yet begun to make decisions about how their current policies and procedures will need to change. State policymakers in many states will likely be faced with enacting new legislation to align their current accountability systems with the federal legislation.

In the current study, we additionally examined states’ Title I accountability systems and their measures of AYP that are required by federal legislation. When we started to collect information from states, we were interested in seeing how their state accountability systems were related to the Title I accountability system. As we collected data, not only were states in flux with respect to their accountability systems, but federal clarification through regulations was in process as well. In many states accountability criteria were undergoing revision, and in several others AYP was being altered in order to include all students, not just those of low socioeconomic status.

With changes occurring across the country, the data included in our report admittedly represent a snapshot in time of what states were currently doing or planning to do. Nevertheless, it is an important snapshot to take, as states proceed in their implementation of standards-based educational systems.

Method

In order to obtain information on state accountability systems, we used the CCSSO Web site (http://www.ccsso.org) and selected "State Education Agencies" from the menu options. This provided a direct link to each state’s Web site, where we conducted a careful analysis of the accountability information available. If a state agency site had a search engine, we entered words such as "accountability," "assessment," "accreditation," "adequate yearly progress," "Title I," and a variety of other terms in an attempt to obtain as much information as possible.

After we gathered initial information from states’ Web sites, we emailed state accountability and Title I directors in December 2001, asking for help in locating pertinent accountability information on their state’s Web site. A second request was made in January 2002 to those who had not already responded.

The following criteria were used to determine whether an "accountability system" existed within each state: (1) schools or districts receive a state-determined rating that is tied to rewards, sanctions, or assistance from the state, and (2) the rating is based, at least in part, on student achievement outcomes. In order to be included in our analysis, it was not necessary for the accountability system to be currently in place; every effort was made to include information for systems that were being developed. In order to determine how students with disabilities are included in each system, we relied on information directly available on the state Web site and in publicly available documents provided by state department employees.

After summarizing the information available for these systems, a copy of the draft tables containing the pertinent data was emailed to the individual states in late May and early June, 2002. We sent a copy to both the accountability/assessment person and the Title I contact person. Changes in tables were made according to new information that states provided. Stringent criteria were used for any changes to be made to information we found on the inclusion of students with disabilities. Specifically, we required the state to provide the actual reference to their inclusion on the Web site or in a publicly available document. We required clear documentation that students with disabilities are included in accountability determinations, and not just that they are included in statewide assessments.

Results

As was noted in the previous search of accountability systems (Krentz et al., 2000), the amount and depth of state accountability system information available on state education agency Web sites varied greatly from state to state. Some states had very specific information available about how accountability ratings were calculated; others had very limited information on the accountability system in general.

Status of Accountability System

Table 1 represents a snapshot in time of the status of states’ accountability systems. When creating Table 1, we looked online to determine whether a state had a state accountability system entirely in place, or in the process of implementation. In some situations, it was very difficult to determine the implementation status of a state’s accountability system. In those circumstances, we chose a state’s category of implementation status using the information we had available, realizing that our decision may not have been entirely accurate. For this table, we focused on the stage of implementation of an accountability system for all schools and students, and placed in a separate column the status of accountability for students and schools receiving Title I services, when this status was different from that for the overall accountability system. In general, we identified states at many different stages of implementation. Because accountability systems are constantly being modified, and because legislation and rulemaking at both the state and national levels is changing, every state could potentially end up in one of two columns – either "in development phase," or "currently in place, but being revised."

Table 1. Current Status of Accountability Systems

State	Entirely in Place	Currently Being Phased-in	Currently inPlace, but Being Revised	In Development Phase	Status of Title I Accountability System (if different from overall accountability system)
AL			X^a,b
AK				X
AZ		X^c
AR		X
CA	X
CO	X
CT	X
DE		X
FL	X
GA				X	Title I system is in place.
HI				X	Title I system is in place, but being revised to align with state accountability.^b
ID				X	Title I system is in place, but being revised to accommodate new AYP measures.
IL			X^b
IN				X^d	It appears that the school Title I system is in place.
IA	Accountability system did not meet criteria for inclusion in this report.
KS	X				Title I system is currently in place, but being revised.
KY	X
LA			X^b
ME					Title I AYP is being phased in.
MD			X^b,e
MA	X
MI				X	Title I AYP is in place, but being revised.
MN					Title I AYP is in place.
MS				X^f
MO	X
MT	X^g				Title I accountability system appears to be in development.^h
NE	Accountability system did not meet criteria for inclusion in this report.ⁱ
NV	X
NH	Accountability system did not meet criteria for inclusion in this report^j
NJ			X^k		Title I AYP system is in place, but being revised.
NM	X
NY	X
NC	X
ND				X^g	Title I AYP system is in place.
OH			X^k
OK	X
OR		X
PA			X^b
RI			X^b
SC		X			Title I AYP system is in place.
SD				X	Title I AYP system is in place, but being revised.
TN	X
TX			X^l
UT					Title I AYP system is in place^m
VT			X^b,n
VA		X
WA	X
WV			X^k
WI	X
WY	X^g				Title I system is in place, but being revised.
Total	16	6	11	9

^aA state department contact indicated that the new accountability system will be drastically different from that of the past.
^bFor this report, the system currently in use or being phased-in is described.
^cWorking on matching state system with federal Title I system.
^dIndiana’s new school improvement and performance categories were used for the purpose of this report (not the determination of accreditation status, which includes compliance with other state rules)
^eBased on “Recommendations of the Accountability Task Group” (2001), it appears that the current accountability system is being revised.
^fEmail from state department suggests that the new growth model accountability system will be in place after the 2002-2003 school year. For this report, included information available on performance accreditation.
^gAn accreditation system is in place/development that considers student achievement outcomes; however, this accreditation is primarily based on indicators other than student achievement outcomes. Number totals do not include these states.
^h Web documents suggest that MT’s Title I system did not meet federal requirements; therefore, they are working on a compliance agreement that would give MT more time to develop their Title I accountability system. For the purpose of this report, not enough information was available to adequately describe Montana’s Title I accountability system.
ⁱ Schools are recognized for exemplary assessment quality and performance in a report of the state.
^jThere is very limited information suggesting that a program will be developed.
^kFor this report, the new system is described.
^lBased on article from the Houston Chronicle.
^mIt appears that there are school level standards, with state established acceptable levels of performance, but the requirements to meet established levels were not indicated, and it is not clear that these results are used for rewards, assistance, and/or sanctions.
ⁿCurrent system is phasing in accountability decisions.

As well as being at various stages of implementation, states also use various strategies to meet full implementation of their accountability systems over a given period of time. Some states use a strategy of phasing-in various grade level assessments to the accountability system to over time. For example, Louisiana started testing in high school in grades 9 and 10, and then later added grade 11 testing. Similarly, Ohio plans to implement an accountability system by phasing in tests for different grade levels at different points in time, although their system is still in the process of being revised.

Other states set a time frame for implementation, establishing a trial period before the testing became "high stakes." For example, Arkansas began data collection in the year 1998-99. Actual points for rewards and sanctions were not assigned until 2001-2002, and then only for certain indicators. Additional rewards and sanctions are scheduled to be phased in over time. Similarly, Vermont is phasing in accountability decisions, which are not scheduled to occur until the 2002-2003 school year.

States such as Delaware are in the process of phasing in various content areas for accountability purposes. This is consistent with the federal ESEA provision that requires states to begin assessing Reading and Math in grades 3 through 8 and add assessments in science in subsequent years. Finally, there are states such as Alabama that have created and implemented a system with a preliminary set of indicators, and are still in the process of determining which indicators will ultimately be included in the system.

States have also used a variety of strategies to meet the requirements of Title I and earlier ESEA legislation. Some states had very distinct Title I and statewide accountability systems. Still others incorporated the Title I system into their statewide accountability system. Many states had similar state accountability and Title I systems. In these cases, either the statewide accountability system provided the framework for the Title I system, with an additional labeling system for Title I schools, or a school or district could be identified as not meeting Title I AYP via the statewide accountability formula. Communication with state educational agencies indicated that many states are working on aligning pre-existing or new state accountability systems with Title I requirements. There are also some states that had Title I accountability systems in place, but were in the process of developing an accountability system for non-Title I students. Trying to determine whether states had two systems they were hoping to merge, or whether they were developing a new system to take the place of the old, or some other approach or combination of approaches was virtually impossible. Many states were in flux, and most were attempting to figure out whether what they were doing would meet federal requirements.

Several states have, or are developing, accreditation systems that address student achievement or improvement (Montana, North Dakota, Wyoming); however, student achievement is only a very small factor in determining the overall accreditation status. Therefore, these systems were not included for the purpose of this report. It appears that Utah and New Hampshire may be developing accountability systems for all students in the future, but extremely limited information of this was available. Iowa has a system that is administered primarily by area agencies, and not by the state. Finally, Maine, Minnesota, and Nebraska also do not appear to have or be developing accountability systems that meet the criteria we used. However, when relevant information was available on these states’ Title I accountability systems, this information was described, with footnotes explaining that the information provided pertains to the state’s Title I accountability system.

Accountability Indicators

Table 2 displays the indicators used to determine school or school district labels and consequences. The indicators represented in this table are those that are actually counted in the state’s accountability measurement. Many states have "report card" indicators that are used to provide information to parents, policymakers, and the community about particular characteristics of a school or district. Unless indicators were used for more than simply "information," we did not include them in this table.

Some states have both a rating system and an accreditation system that incorporates multiple indicators in addition to student test scores. For these states, we identified indicators only for the primary system in Table 2, but provided corresponding footnotes to explain indicators used for the additional system. For instance, Colorado primarily uses an accreditation system to hold schools accountable, although there is an "academic performance rating" system that takes into consideration a subset of the indicators used for the accreditation system. Several states (including California, New York, North Carolina, South Carolina, Texas, and Virginia) have different accountability indicators used for special schools (e.g., alternative, career/technology, or special education schools), or allow these schools to choose additional indicators for accountability purposes.

Table 2 includes both Title I AYP and general accountability indicators; indicators used solely for Title I AYP determination are noted in the table. Overall, student assessment scores are the indicators used most often in the determination of school consequences. In addition, they are frequently the most heavily weighted components of school accountability formulas. For many states, both test performance and achievement growth as measured by tests play a significant role in school accountability, although the extent to which achievement growth is weighted in the formula varies from state to state. Attendance and dropout rate are the next most commonly used indicators (20 states); assessment participation rate, graduation rate, and indicators of school safety are also frequently included in measures of school accountability. Participation rate is included as an indicator in a variety of ways. In several states (Florida, North Carolina, and Pennsylvania), a school must test a certain percentage of students in order to be eligible for rewards. In other states, students who did not participate are automatically given a score of "0". Maryland uses a unique method to ensure that participation rate is appropriately accounted for as an indicator in the accountability system. In this system, a school’s participation rate can fluctuate over time; however, the average "Level Not Determined" (LND) cannot be higher than 10% across five years, and in no single year may there be more than 14% in the LND category. Finally, in the most recent year the LND category must contain 6% or less of students.

In some state systems, review of assessment scores is the first step to determining labels or consequences for schools or districts. If schools fail to meet necessary achievement levels, then other indicators are additionally reviewed. In other states, cut-off levels for each indicator must be met for a school to receive the highest ranking. Still other states maintain a point system for determining school labels; schools receive a set amount of points for each indicator, and ratings are then determined based on total points for the school or district.

In some state accountability formulas, special attention is placed on particular student types. For instance, Alaska is considering weighting low-performing students’ scores twice as much as average-to-high performing students’ scores in the accountability formula. Many other states have formulas that specifically target low-performing students, minority students, or other students considered to be at-risk. Student mobility is an issue that is addressed in different ways across state accountability systems. In some states it is clearly indicated that a school is not responsible for the scores of students who enter the school after a given point in the school year.

Table 2. Indicators Used to Determine School/District Labels and Consequences

Assess Scores

Assess Score Growth

Attend

AP Info

ACT/ SAT

Dropout Rate

Enrlmnt

Grad
Rate

Retention Rate

Student/
Teacher Ratio

Suspension Rate

Staff Indctrs

Assess Partic Rate

Use of Resource Indctrs

Safety

Other indicators

AL

X

X

Other indicators will be added in the future according to state contact person

AK

X

X

X

X

X

Post-grad success, Local assessments

AZ

X

X

X

X

School site visit for schools labeled under performing two consecutive years

AR

X

X

X

X^a

X^a

X

X

Other school selected indicators, school narrative

CA

X

X

X^b

X^b

CO^c

X

X

X

Compliance with Safe Schools Act and the Colorado Literacy Act, having an educational improvement plan, evidence of reporting other information

CT

X

X

DE^d

X

X

X

FL

X

X

X

GA

X

X

X

HI

X

X

X^e

X

X

Other indicators may be added in future, according to state contact person

ID

X

X

X

X

X

X

X

X

X

X

IL

X

X

IN

X

X

X

X

Percent of graduating students who earn the academic honors diploma and Core 40 diploma.

IA

Accountability system did not meet criteria for inclusion in this report.

KS

X

X

X

X

X

X^b

X

Mastery of algebra, passing advanced math and science courses, local assessments

KY

X

X

X

X

X

Successful transition (for high school)

LA

X

X

X

X

ME^f

X

X

Completion of portfolio assessments, performance on local measures

MD

X

X

X

X

MA

X

X

Other indicators (dropout rates, attendance, improvement trends) may be used to determine whether full review of underperforming school is necessary.

MI

X

X

Indicators of instructional quality, indicators of learning opportunities, indicators of engagement

MN^f

X

X

MS

X

X

MO^g

X

X

X

X

X

X

X

X

X

X

Student/Administrator Ratio, Advanced Courses, Vocational Courses, Vocational Education Follow-Up, College Placement, GPA, Climate, Instruction, Preschool, Parent Communication, Community Involvement, Health Services, Foods Program, Transportation

MT

Accountability system did not meet criteria for inclusion in this report.

NE

Accountability system did not meet criteria for inclusion in this report.

NV

X

X^e

X

X

NH

Accountability system did not meet criteria for inclusion in this report.

NJ^h

X

X

X

X

X

X

X

Provide board approved program of guidance and counseling services

Status of mandated programs/services, budget, external audits, transportation, health, other reports, substance abuse program, offering of other programs

NM

X

X

X

X

X

X

Parent/Community Involvement, Compliance with State Board of Education requirements, federal regulations, and state statutes, progress toward the EPSSgoals

NY

X

X

X

Certain exams, such as SATs and AP exams, may be used as approved alternatives to Regents High School examination scores.

NC

X

X

X

X

College University Prep/College Tech Prep Component

ND^f

X

X

OH

X

X

X

X

OK

X

X

X

X

X

X

X

X

X

Vocational-technical program participation and completion rates, college entrance and preparatory test-taking rates, parent involvement rates, test results in the context of socioeconomic status and finances of the school district, other indicators reported through Oklahoma Education Indicators Program

OR

X

X

X

X

X

Local data used for k-2 AYP

PA

X

X

X

Xⁱ

X^e

X

Job-related placement for Vocational-Technical Schools. Student technology literacy^e, percent of migrant students enrolled in “schools in need of improvement”^e, percent of LEP students who have obtained English proficiency^e

RI

X

X

X^b

X^b

X^b

X^b

X^b

X^j

X^b

(Safety is part of school climate and school work climate.)

SC^k

X

X

X

X

X

X^e

X

X

X^j

Parent Involvement, External accreditation, Enrollment, placement, and graduation rates for Career and Technology centers, Other factors promoting or maintaining high levels of achievement and performance, criteria appropriate to each school’s mission (used for special schools), School Readiness^e

SD

X

X^e

More indicators may still be added.

TN

X

X

X

X

TX

X

X^l

X^l

X^l

X^l

X

Advanced course completionⁿ

UT^f

X

X

VT

X

X

X

Local assessments

VA

X

X

X

Must also meet compliance with promotion/retention policies, course offerings, and staffing regulations.

WA

X

X

WV^m

X

X

X

X

Random audits are also completed to check compliance with 97 standards.

WI

X

X

X

WY^f

X

X

46

45

20

2

5

20

3

12

4

4

3

7

14

2

9

^aOne of the goals is defined as: “At least 99% of secondary students will remain in school to complete the 12^th grade”
^bThese indicators will be used to determine school labels in the future.
^cThis reflects Colorado’s district accreditation system. An Academic Performance Rating is also determined for schools, which takes into account test scores and ACT scores.
^dThere also is an accreditation system, which appears to be based primarily on the school accountability system described above.
^eThese indicators represent those used for Title I AYP, but not used for the overall accountability system.
^fThese states currently only have Title I AYP systems, and do not have overall accountability systems that meet our definition. The indicators marked for these states represent those used for Title I AYP.
^gThis reflects Missouri’s accreditation system. A performance rating system is also in place, which takes into account student achievement scores and graduation rate in determining whether a school qualifies as “academically deficient” or “concerned”.
^hDistricts are given two labels (Group and Level), and the evaluation is based on whether schools have met certain requirements. All potential indicators are included in this chart. An alternate accountability system is used for Group 3 districts.
ⁱSchools need enrollment data on the web to receive awards.
^jAssessment participation rate takes into consideration total enrollment.
^k The indicators used depend on the type of school; this represents any and all indicators that might be used for a school.
^lThese are used for additional acknowledgements (gold performance acknowledgement), not overall accountability rating.
^mDropout and attendance data are used in the current system, and it is assumed this will also be a component of the new system; however, this assumption is not documented.

Accountability Growth Indicators

Table 3 reflects the types of growth indicators included in accountability systems. In many states, a growth component is added only if a school or system does not meet a set criterion level. In other states, all schools, regardless of achievement level, need to demonstrate progress. Several states require incremental growth in achievement scores such that all schools reach a goal of having a target percent of students scoring in the proficient range by a particular year. Under these circumstances, lower achieving schools are required to make greater progress than higher achieving schools.

Of the three categories of growth indicators represented on this chart, school achievement growth is the mostly widely used. In this type, a particular school, grade level, or subgroup’s score is compared from one year to the score for the next year’s subgroup in order to determine "achievement growth." This is the easiest score to determine, but does not take into consideration factors such as changes in the school population, and the challenge of drawing comparison between distinct cohorts of students. This type of growth measure is required by Title I.

A second form of growth examines cohort achievement. In this case, "growth" is determined on a longitudinal basis, looking at how a specific group of students performed over time. It is sometimes referred to as a "value-added" approach. Similarly, an individual achievement growth component breaks down the data to an individual student level, and follows individual student performance growth over time.

In some states, the school achievement growth factor is used to determine Title I AYP, but may not yet be determined for the non-Title I population. In other states, it has been decided that they are going to include a growth component, but the specific type of growth has not been determined at this time. Several states include multiple types of growth in their accountability systems.

Table 3. Types of Growth Indicators

State	School Achievement Growth	Cohort Achievement Growth	Individual Achievement Growth	Growth Type To Be Determined
AL	X
AK		X
AZ	X		X
AR	X	X
CA	X
CO	X^a
CT	X^b
DE	X
FL	X^b		X^c
GA	X^d			X
HI	X
ID^e	X	X
IL	X^f
IN	X^{a, d}		X^g
KS	X
KY	X
LA	X		X^h
ME	X^b,d
MD	X
MA	X
MI	X		X
MN	X^d
MS				Xⁱ
MO^j	X^d
NV	X^b,^d
NJ	X
NM		X
NY	X^b
NC	X	X	X
ND	X^b,d
OH^b,k
OR	X
PA	X
RI	X
SC	X^d		X
SD	X^d
TN	X^b		X
TX			X^l
UT	X^d
VT	X
VA	X^b
WA	X
WV^m	X		X
WI	X^b
WY	X^d
Total	40	5	10	2

^aOther growth types may be chosen by districts to demonstrate progress.
^bNot used for schools that already meet cut-off level.
^cEventually the system will look at individual student learning gains.
^dThis is used in determining Title I AYP
^eUnclear whether will use individual achievement growth.
^fAll schools receive an improvement rating (See Table 5). Schools not meeting 50% of students proficient must demonstrate yearly increases in percent proficient.
^gImprovement will be measured by increase in achievement of a non-mobile cohort group of students (tracked by identification number) as they progress through school.
^hIndividual growth of students demonstrating very low achievement (students who are retained, and students taking out-of-level tests) is included in determining label. It is not clear whether the progress of other individual students is included in the label determination.
ⁱUnclear from available information which growth type they will use (in development).
^jGrade span growth is measured (3-5, 6-8, 9-11).
^kSchools not meeting the minimum number of indicators must develop three-year continuous improvement plan, and must make satisfactory progress with this—these are locally determined, and so may differ across districts.
^lThis is used for determination of additional acknowledgement.
^mNew accountability system will include annual changes in students’ scores, trends in scores, and goals for schools in average scores.

AYP Determination

Table 4 displays information on how states determine AYP. At the point that we collected information, many states made distinctions between their Title I AYP and other accountability systems. In many cases, information was only available on former definitions of AYP. Current legislation has caused many states to begin revising their definitions of AYP; therefore, we dated the information in order to clarify when the definition described was applicable.

In some states, AYP was a requirement for all low-performing schools. In other states, AYP was only a requirement for those schools and districts receiving Title I funds. Some states had synonymous state and Title I accountability systems in which AYP was demonstrated by the same indicators as those used for the state accountability system. In other cases, the Title I AYP system was distinct from the other accountability system.

The definition of "AYP" differed from state to state. In some states AYP is met when schools demonstrate that a certain percentage of students with disabilities meet proficiency requirements on a state test. In other states, AYP is not required among schools that meet these proficiency requirements, and only among those that do not meet the threshold requirements.

Most states require an increase in the percentage of students scoring within a proficient range on a statewide test in order to meet AYP requirements. In some definitions, all schools not meeting the threshold percentage must each make the same percentage improvement on a yearly basis. In other states, the required percentage improvement depends on the current status of the school. Formulas have been developed to determine the amount of progress necessary for the school to meet a target set for the future. The required AYP, then, depends on the current distance from the target. Several states use a multiple-year average to determine the baseline for a given year, in order to account for year-to-year fluctuations in the student population.

Several states use a school index to measure AYP. The index is a numerical value assigned to a school that represents a calculation based on student performance, and in many situations includes additional indicators such as dropout rate and attendance. The percent of students

Table 4. State Adequate Yearly Progress (AYP)

State

Year of Document

Increase in Mean or Median Score

Percent of Students Proficient

Increase in Percent of Students Proficient^a

School Index

Growth in School Index

Additional Comments

AL^b

2001

AK^b

2001

Indexes will include dropout rate, percent graduating, post-grad success, and local assessments.

1999

May look at other data (criterion-referenced tests, portfolio assessments, norm-referenced tests, district writing assessment, District Achievement Plans) to determine AYP.

AR^b,c

1999

Title I AYP status is determined based on both performance and percentage of improvement.

2000

A school demonstrates adequate progress by meeting or exceeding its API growth target, which includes multiple indicators. A school must meet its five percent schoolwide growth target and also meet the comparable growth targets for all numerically significant subgroups in the school.

2001

Review of other data may also be used when a school does not demonstrate AYP. This may include the following: percent of ELL, high proficiency rate, changes in enrollment, etc.).

2001

2000

FL^b

2002

If 50% of the lowest performing students (lowest 25% in the school) make learning gains, the school demonstrates adequate progress. This requirement is for schools designated as performance grade "C" or above.

1998-1999

form

AYP appears to be based on an 8% decrease in the number of students scoring in the 1^st to 39^th percentile and meeting a goal for increase in the normal curve equivalent average.

1999

For 1999, it appears that AYP was met if a school showed progress on three out of four indicators, one of which could be a school-selected indicator. In addition, schools must meet a goal of 95% attendance rate or at least a gain of 2%.

Accountability system did not meet criteria for inclusion in this report.

2000-2001

In revision stage; new measures of AYP are being developed.

2001

2001^d

2001

A point system is currently used to determine AYP. Schools must obtain 62% of the points possible for their school configuration. Points are awarded for the following: 1 pt. Increase in Building Index, 99% of eligible students taking the assessment, 70% of students reach standard on local performance assessment, and reductions in percentages of students scoring below cut-off levels.

2001^d

Index includes dropout rate, retention rate, attendance, successful transition, in addition to assessment scores.

LA^b

1999-2000

Indexes include attendance and dropout rate, in addition to assessment scores.

2002

Schools also demonstrate progress when 80% of students exceed the benchmarks according to local measures, and when 80% of Title I students have successfully completed three portfolio tasks in the assigned content areas.

MD^b

2001

Index includes dropout rate and attendance, in addition to assessment scores. Schools are also identified if they have had declining scores for two consecutive years. Schools must improve scores for two years to exist "list."

MA^b

2002^e

1996

2001

X^f

MS^b

2001

Consequences occur for schools when they do not meet their assigned yearly growth expectation^g, and if they have a specified percentage of students functioning below grade level.

2001

If the percent of students in "Level Not Determined" is 10% or more (see text for more detailed information), the district or school is designated as not meeting adequate yearly progress.

Accountability system did not meet criteria for inclusion in this report.

2001

Accountability system did not meet criteria for inclusion in this report.

2000

NM^b

2002

2001

Schools that do not meet the state standard must set targets such that they close the gap between current performance and the state standard by 15% each year. This index includes dropout rate.

2001^d

X^h

Schools with a low percentage of students meeting standards must meet their growth standard, which is based on year to year growth for the same students over time, and is measured by a composite of the percentage of students meeting proficiency on a variety of different tests and indicators. Schools with a very small percentage meeting proficiency may not be considered to meet AYP even if they meet their growth standard.

2001^d

School must be identified for school improvement if have mean composite score at partially proficient level for 2 consecutive years and have failed to make 2.0 target gain in total battery score, or if have mean composite score at novice level for 2 consecutive years.

2001

Has not yet determined how AYP will be calculated for new system.

No definition of AYP was identified.

2002

Combined index includes growth and participation components.

2002

Currently being revised. Will take into account percent of Title I students making adequate yearly progress, percent of migrant students enrolled in schools in need of improvement, percent of students meeting standards for technology literacy, percent of LEP students attaining English proficiency, percent of 3^rd grade students reading at grade level, and graduation rate.

2002

2000-2001^d

Schools that meet standard for percent of students meeting standards must not drop by 3% or more annually. In addition to test scores, school readiness and promotion/retention rate is used.

2001

Also must reduce dropout rates and meet expected value-added target.

2000

Dropout rates are also included in determination of AYP, which is met when a school receives an acceptable rating.

2001

Xⁱ

2001-2002^c

Participation rate and local assessments are also considered.

2001-2002

2001

Also based on meeting Reading and Mathematics Improvement Goals for Grades 4, 7, and 10.

2001-2002^d

In development to correspond with state accountability system.

1999

A Substantial and Continuous Improvement (SCI) index is used at the first level of analysis. Other levels of analysis include first and second grade assessment performance, progress of Title I students, and the movement of students into more advanced categories.

Total

aOr decrease in percent of students not proficient.
^{bThe words "Adequate Yearly Progress" were not used; however, it was

assumed that AYP will coincide with the state accountability system for all

schools, and so state accountability information was coded in this table.}
^{cAlthough "AYP" words were not used, this information was verified by state

department contact.}
^{dThis information was provided in 2001/2002 by a state department contact.}
^{eExact date was not identified on documents, but the connection between

Title I AYP and accountability system is apparent on a document that describes

the process used for FY2003.}
^{fSchools that meet the requirement of having an average scaled score of

1420 do not need to show improvement.}
^{gAnnual growth expectations were not defined.}
^{hSchools must meet the growth standard, which is described in the table.

This does not exactly represent "growth in school index" as described in the

text, but represents something very similar to "growth in school index."}
^{iLimited information was identified on AYP determination. It appears that

the school median percentile score, and increase in this measurement are used to

determine AYP.}

Sources of Indicators

In a majority of states, the indicators used for accountability are determined at the state level. However, there are some states that require local school districts to determine some of their own indicators, and therefore use a mixture of state and local indicators (see Table 5). If states only use local indicators, and do not identify which indicators are required, they are not included in the information represented in Table 5.

Accountability Labels

States use a variety of labels to describe school or district status. There are schools and school districts across this country that have been determined to be "distinguished," "successful," "excelling," "superior," "commendable," and "exemplary." There are also schools and districts that have been labeled "deficient," "under performing," "failing," "marginal," and "academic alert." Table 6 represents the broad array of labels given to schools and districts across the country. Some states have three levels of performance; others have four or five levels. Some states have an index score, and some use absolute grades or progress grades. Others are still determining the labels they will eventually attach to their schools.

Many states have either separate or additional labels applied to Title I schools. These labels are found in Table 7. Again, states use a variety of labels to indicate the status of Title I schools.

Table 5. State vs. School/District Determined Indicators

State	All State Determined Indicators	Mix of State and District/School Determined Indicators
AL	X
AK		X^a
AZ	X^b
AR		X
CA	X
CO		X
CT	X
DE	X
FL	X
GA	X
HI	X^b
IA	Accountability system did not meet criteria for inclusion in this report.
ID	X^b
IL	X
IN		X^c
KS		X
KY	X
LA	X
ME		X^d
MD	X
MA	X
MI	X
MN	X^d
MS	X
MO	X
MT	Accountability system did not meet criteria for inclusion in this report.
NE	Accountability system did not meet criteria for inclusion in this report.
NV	X
NH	Accountability system did not meet criteria for inclusion in this report.
NJ	X
NM		X^e
NY	X
NC	X
ND	X^d
OH	X
OK	X
OR	X^f
PA	X
RI	X
SC	X
SD	X
TN	X
TX	X
UT	X^g
VT		X
VA	X
WA	X
WV	X
WI		X^h
WY	X^d
Total	37	9

^{aLocal assessments may be added as

indicators in the future.

^{bCurrent Title I AYP allows use of local measures.}

^{cSchools can determine specific tests to be used at particular grade levels}

^{dThis represents Title I AYP determination.}

^{eIt appears that EPSS goals may not be determined by the state.}

^{fLocal assessments can be used for k-2 AYP}

^{gLimited information was available on Utah’s AYP system; it is assumed that

this is state determined.}

^{hThe district can cite other evidence of continuous progress if the

department finds that it is not meeting improvement requirements. The state may

then decide to withdraw the identification of not meeting improvement.}}

Table 6. Labels Given to Schools/Districts

State	Label
AL	Academic Clear, Academic Caution, Academic Alert (1, 2, & 3)
AK	Distinguished, Successful, Deficient, In Crisis
AZ	Excelling, Improving, Maintaining, Under Performing, Failing
AR	High Priority, Alert, Low Performing, Academic Distress (I, II, III)
CA	Academic Performance Index Score (API) (200-1000)
CO^a	Accredited, Academic Watch, Academic Probation, Not Accredited
CT	A numerical index ranging from 0 to 100 is given to each school; schools with a low index are classified "priority" schools.
DE^b	Superior, Commendable, Under School Review, Unsatisfactory
FL	A: Making excellent progress, B: Making above average progress, C: Schools making satisfactory progress, D: Schools making less than satisfactory progress, F: Failing school
GA	Absolute Grades: A,B,C,D,F Progress Grades: A,B,C,D,F
HI	No labels identified yet, although "on the right track-give more time" and "give customized, mandatory assistance" have been noted as possibilities.
IA	Accountability system did not meet criteria for inclusion in this report.
ID	Proposed labels for system in development: Distinguished, Achieving, Improving, Needs Improvement
IL	Levels 1-6: 1= exceeds standards, 2=’high’ meets standards, 3=meets standards, 4=approaching standards, 5=limited success in meeting standards, 6= little/no success in meeting standards. Also, there are Academic Watch and Warning lists for schools that do not demonstrate adequate yearly progress. Improvement Labels: Improving, Stable, and Decreasing
IN	Exemplary, Commendable, Adequate, Marginal, Unsatisfactory
KS	A Candidate, Accredited, Accredited Conditionally, Denied Accreditation
KY	Meets Goal, Progressing, Needs Assistance, Commonwealth Pace-setter. There are three levels of assistance (Level 1, Level 2, and Level 3).
LA	Performance Labels: School of Academic Excellence, School of Academic Distinction, School of Academic Achievement, Academically Above the State Average, Academically Below the State Average, Academically Unacceptable Growth Labels: Exemplary Academic Growth, Recognized Academic Growth, Minimal Academic Growth, No Growth, School in Decline Corrective Actions I, II, III
ME	No labels given to all schools (see Table 7 for Title I AYP labels)
MD	An "SPI" is calculated for each school. This numerical value provides an estimate of the school’s performance on indicators. Excellent, Satisfactory, Not Met labels are also given to schools based on assessment results.
MA	Performance Labels: Very High, High, Moderate, Low, Very Low, Critically Low Overall Improvement: Exceeded, Met, Approached, Failed to Meet
MI	A, B, C, D, F
MN	No labels given to all schools (see Table 7 for Title I AYP labels)
MS	Superior-Performing School, Exemplary-Growth School, Priority School, (other designations to follow)
MO^c	Accredited, Provisionally Accredited, Unaccredited
MT	No labels given to all schools (see Table 7 for Title I AYP labels)
NE	Accountability system did not meet criteria for inclusion in this report.
NV	Exemplary Achievement, High Achievement, Needing Improvement
NH	Accountability system did not meet criteria for inclusion in this report.
NJ	Group 1, Group 2, Group 3 (Alternative Evaluation), Level I, II, III
NM	Level 1 (Exemplary, Exceeds Standards, Meets Standards), Level 2 (Probationary Approval), Level 3 (Disapproved)
NY	"Schools Under Registration Review"(SURR). A school is categorized as "has met all applicable standards," "is below one or more standards", or "is farthest from meeting one or more standards."
NC	School of Excellence, School of Distinction, School of Progress, Priority School, Low-Performing School
ND	No labels given to all schools (see Table 6 for Title I AYP labels)
OH	Excellent, Effective, Continuous Improvement, Academic Watch, Academic Emergency
OK	Low-performing, High Challenge
OR	Unacceptable, Low, Satisfactory, Strong, and Exceptional.
PA	"Governer’s School of Achievement", "Closing the Academic Achievement Gap", and the "Maintenance of High Standards (MHS)" awards
RI	Performance Categories: High, Moderate, Low Improvement Categories: Math & English/Language Arts, Math, English/Language Arts, Writing, None "in need of improvement"
SC	Performance Ratings: Unsatisfactory, Below Average, Average, Good, Excellent Improvement Ratings: Unsatisfactory, Below Average, Average, Good, Excellent Other Labels: Receiving gold award, Receiving silver award
SD	Currently just use "Y" or "N" for AYP. Later, schools may go into School Improvement.
TN	Low-performing, "On Notice," School Improvement Grades (A, B, C, D, F) will also be designated for each indicator among schools.
TX	Exemplary, Recognized, Academically Acceptable, Academic Unacceptable/Low Performing; Schools are also eligible for an additional acknowledgement: Gold Performance Acknowledgement.
UT	No labels given to all schools (see Table 7 for Title I AYP labels)
VT	"School for mandatory technical assistance", Numerical Status and Change Index scores are reported
VA	Fully accredited, Provisionally accredited/Meets state standards, Provisionally accredited/Needs improvement, Accredited with warning, Accreditation denied, Accreditation withheld/Improving school near accreditation
WA	Schools in school improvement, focused assistance schools (subset of schools in school improvement)
WV	Exemplary accreditation, Full accreditation, Conditional accreditation, Temporary accreditation, and Seriously impaired status
WI	Schools are individually given a numeric continuous improvement indicator, which they must meet, or otherwise be labeled a "school in need of improvement."
WY	No labels given to all schools (see Table 7 for Title I AYP labels)^d

^aAcademic Performance Ratings are also determined based on test scores alone. These labels are as follows: Excellent, High, Average, Low, Unsatisfactory
^bAccreditation labels are Superior Accredited, Accredited, Accreditation Watch, and Not Accredited
^cConcerned school and Academically deficient school are labels that are also used.
^dThere is an accreditation system, but this is based on student performance according to local standards, and therefore is not described for this report.

Table 7. Additional Labels for Title I Schools

State	Label
AL	School Improvement Year 1, Year 2, Year 3, or Year 4
AZ	In need of school improvement
AR	On "School Improvement Alert" list, Corrective Action
CA	Program Improvement (PI) school
CO	Distinguished, Identified for school improvement, In school improvement, Corrective action
CT	School "in need of improvement"; Level 1, Level 2, Level 3
DE	Title I Distinguished School, Identified for School Improvement
GA	Distinguished, Adequate Yearly Progress, Needs Improvement Adequate Yearly Progress, Needs Improvement
HI	Corrective Action School
ID	Distinguished, Identified for School Improvement
IL	Title I School Improvement
IN	Level I (Warning/School Improvement), Level II (Adequate Yearly Progress), Level III (Advanced)
KS	Title I school improvement, "make AYP"
KY	Title I School Improvement
ME	Meet the Standards, Priority Schools, Needs Improvement Schools
MD	Needing improvement, Under Corrective Action
MA	In need of improvement, Identified for corrective action
MI	Identified for improvement, Identified for corrective action
MN	Performance Labels: Meeting state expectations, Not meeting state expectations Growth Labels: Making progress, Not making sufficient progress, Needs assistance
MS	Distinguished, Needs Improvement, School Improvement Awardee
MO	Not Making Adequate Yearly Progress, Making Adequate Yearly Progress
MT	School identified for improvement
NV	Meet AYP, Not Meeting AYP (School Improvement)
NJ	Category I, II, III, IV, V, VI
NY	Met Standards, Met Adequate Yearly Progress Targets, Schools in Need of Improvement, Corrective Action Schools
NC	In School Improvement, Corrective Action
ND	Distinguished, Identified for Program Improvement
OH	Identified for School Improvement, School Improvement I, School Improvement II, Corrective Action
OK	School Improvement (SI)
OR	"In program improvement"
PA	School Improvement
SC	Meeting adequate yearly progress, Not meeting adequate yearly progress
SD	Distinguished Schools, School Improvement Schools, Schools in Alert Stage
TN	Title I Distinguished School
TX	Distinguished, Honored, Commended
UT	School Program Improvement (SPI)
VA	Title I School Improvement School
WA	"Not met adequate yearly progress," "identified for improvement"
WY	"In need of school improvement"

Inclusion of Students with Disabilities

One purpose of this study was to determine the inclusion or exclusion of students with disabilities in school or district accountability systems. Federal law under IDEA requires this inclusion for assessment, and the newly authorized No Child Left Behind Act requires inclusion of all students in accountability systems. After passage of federal legislation requiring the inclusion of all students in accountability systems, several states have undergone revision of their procedures, and many others will be undergoing revisions.

There are a variety of ways that students with disabilities can participate in statewide assessment systems, and these can affect their inclusion in accountability. Some students with disabilities are included in the regular test administration in the same way as most other students. Others participate with accommodations. Some take modified tests or receive non-standard test administrations, and a small percent participate in alternate assessments. Although state policy may clearly indicate the ways in which students with disabilities can participate and be included in state assessment systems, the degree to which scores from these various test administrations are included in the determination of school accountability is often unclear.

Table 8 represents the extent to which states clearly articulate which scores for students with disabilities count in the determination of school labels and consequences. We used very strict criteria in determining whether there was documented inclusion of students with disabilities in accountability systems. In some cases, the information we obtained indicated that "all students" were included, but we could not find evidence of how students with disabilities were included. In the past, we have found that state assessment performance results reported as being for "all students" have not necessarily represented all students with disabilities. Therefore, we considered such statements unclear. Information for this table was obtained by searching the Web sites of the state educational agencies, and from publicly available documents provided by state department contacts. In addition, the table was verified with state departments, who could provide additional documentation.

Altogether, we identified 27 states with publicly available documentation that at least some students with disabilities are or will be included in the determination of accountability labels or consequences. Inclusion of students with disabilities was articulated in a variety of ways. Some state documents explained exactly which scores are or will be included. Others simply specified that all scores for students with disabilities count in the determination of accountability. For only four states (Delaware, Kentucky, South Carolina, and Vermont) was it clear that all students with disabilities, taking either regular, accommodated, modified, or alternate assessments, are to be included in the accountability system (either currently or in the future). Language from these states’ accountability policies is provided in Appendix A.

Table 8. Documentation of Inclusion of Students with Disabilities in Accountability Systems

State	Documented Inclusion					Documented Exclusion			Unclear
State	Reg	Acc	Mod	Alt	Other^*	Mod	Alt	Other^*	Related Doc.^*	SEA contact^**
AL										X
AK	X	X		X		X
AZ	X^a	X^a							X	X
AR					X			X
CA	X	X^b				X^b		X
CO^a	X^a	X^a		X^c	X					X
CT										X
DE	O	O	X	X	X
FL					X
GA									X
HI										X
IA	Accountability system did not meet criteria for inclusion in this report.
ID										X
IL					X
IN					X^a					X
KS					X^a
KY	X	X	X	X
LA	X	X			X			X
ME									X^a
MD	O	X^b				X^b	X
MA									X
MI									X
MN									X^a
MS						X	X
MO				X^d	X
MT	Accountability system did not meet criteria for inclusion in this report.
NE	Accountability system did not meet criteria for inclusion in this report.
NV	O^a	O^a	X^a				X^a
NH	Accountability system did not meet criteria for inclusion in this report.
NJ								X		X
NM	O	X
NY					X
NC	X	X		X	X					X
ND									X^a
OH									X
OK									X
OR	X^e	X^e	X^e		X	X^e
PA					X		X			X
RI										X
SC	X	X	X	X^f	X		X^f
SD										X
TN					X			X
TX				X^g	X
UT									X^a
VT	X	X	X^h	X^h
VA				X	X
WA										X
WV										X
WI	X	X		Xⁱ
WY					X^a,j					X
Total	15	15	6	11	4 (18)^k	5	5	5	9(10)^k	8(15)^k

^{O=this is not documented as being

included; however, given that accommodated and/or modified tests are included,

it was assumed that these administrations are included.

^{*See Table 9.}

^{**See Table 10.}

^{aThis represents information from Title I accountability.}

^{bSome accommodated scores are included; some are not.}

^{cPerformance ratings do not include alternate assessments.}

^{dAlternate assessment is coded as "Level Not Determined" and entered as

such in the accreditation system.}

^{eSome administrations are only included in participation component.

Performance for these administrations will be added in the future.}

^{fAlternate assessments will be included in district accountability, but not

school level accountability.}

^{gWill be included in ratings in the future.}

^{hAlternate assessments, which include modified assessments, are included in

participation component, and will be phased into performance component.}

^{iGiven separate code for accountability purposes, which suggests that only

participation in alternate is included in school accountability.}

^{jThis represents information about system accountability. It is therefore

assumed that this also refers to Title I accountability, which was the only one

of WY’s systems that met our requirements for inclusion (accreditation system

did not focus on student performance results).}

^{kThe number outside of parentheses represents the number of states that

only fall only in this inclusion/exclusion category, and no other categories.

The number in parentheses represents the number of states overall that were

included in this category.}}

We identified 15 states that have clearly articulated in publicly available documents that scores for students receiving accommodations are included in the state accountability system. In some cases, it was clearly articulated that scores for both regular and accommodated test administrations for students with disabilities are or will be included in the accountability system. We assumed that if there was documentation of the inclusion of accommodated scores, results obtained from non-accommodated testing of students with disabilities were also included. For two states (California and Maryland), only those scores obtained with accommodations determined to maintain the validity of the test are included. Other accommodations are not deemed valid, and those scores are excluded.

Modified assessments were less frequently documented as being included in accountability determination. Only six states explicitly stated that modified tests are included, and five states indicated that these scores are excluded from the accountability system.

Eleven states indicated that alternate assessments are or will be included in the determination of accountability. It appears that four states only count the alternate assessment toward the participation component of the accountability system, and do not include alternate assessment performance, although several indicate that performance results on the alternate assessment will be phased into the accountability system over time. Others simply stated that alternate assessments are not included in the accountability system.

Many states did not explicitly state which test administrations are or will be included, but did articulate that students with disabilities are included. These statements are described in Table 9. In some of these situations, statements indicate that growth in the scores for students with disabilities is necessary for accountability purposes. Other policies state that all scores for students with disabilities are included in the accountability determination, but do not necessarily state which test administrations this includes. In one state, it appears that only certain students with disabilities are included, namely those with speech and language disabilities.

Table 9 also indicates other information about the exclusion of students with disabilities from accountability. Students with disabilities in New Jersey and Louisiana, for example, have been excluded in the past, but current documentation or communication with state contact persons indicates that these states are moving toward greater inclusion. Table 9 also contains data from state Web sites that did not meet our criteria for documented inclusion/exclusion.

The amount of searching required to identify how students with disabilities are or will be included in state accountability systems varied greatly from state to state. In Illinois, it is very clear that students with disabilities are included. On the front of the Illinois student assessment Web page, a statement reads "Scores for all students, including students with disabilities and limited English proficiency, are included in the determination of AYP for school accountability." In other states, the information is less easily accessible, and extensive Internet searching was necessary to identify whether documentation was available.

In some situations, documents did not clearly articulate whether students with disabilities are included in the accountability system. In those situations we relied on communication with the state contact persons to obtain the relevant information. This communication is provided in Table 10. Altogether, publicly available documents and communication with state contacts suggests that states are moving toward greater inclusion of students with disabilities in accountability systems.

Table 9. Other Documentation of Inclusion of Students with Disabilities in Accountability Systems

State	Inclusion	Exclusion	Unclear
AZ			X	"All students, including those with disabilities and limited-English proficiency, have the legal right to be included in accountability systems and to be part of the basis for policy decisions that affect them."
AR	X	X		Tier II growth goals include SWD. Tier I performance goals exclude SWD.
CA		X		Score is excluded if tested more than one grade out of level.
CO	X			All students, including SWD, must be considered when setting district goals for closing learning gaps. One year’s increase in student achievement for each year for disaggregated groups, including SWD, is needed.
DE	X			Out-of-level tests are automatically given performance level = 1, and counted in accountability determination.
FL	X			Students with speech/language disabilities included.
GA			X	SWD are included in assessment.
IL	X			Scores for all students, including students with disabilities and limited English proficiency, are included in the determination of AYP for school accountability.
IN	X			Determination of AYP may be analyzed through disaggregation of ISTEP+ data to examine progress of students in special education compared to students in regular education.
KS	X			Students with disabilities are included in 99% participation requirement for determining AYP.
LA	X	X		Awards are provided when school surpasses growth target and shows growth in performance of SWD. Individual progress on out-of-level tests is included, but will receive a score of "0" growth if not tested at least 3 years below grade level. Percent of students with disabilities taken into account when determining growth target. School Performance Score for K-8 schools in 1999 and 9-12 schools in 2001 included only regular education students.
ME			X	Title I AYP system "provides a means to examine disaggregated performance of . . .disabled students".
MA			X	Students with disabilities are included in assessment.
MI			X	All schools, including special education schools will be part of the new plan. For Title I AYP, during the 2001-2002 school year there was "no option of excluding math/science scores of students in special education from school and district summary reports."
MN			X	Title I AYP system does not indicate which scores are included. Students with disabilities are included in assessment.
MO	X			A point is awarded if performance of SWD is increasing or being maintained at a high level in the process section of the accreditation system. Annual AYP evaluation takes into consideration data from all students disaggregated by disability status.
NJ		X		Current documentation indicates SWD are not included.
NY	X			All tested students with disabilities are included in school accountability.
NC	X			Off-level results receive Level 1 or Level 2 scores for determining accountability (Level 3 is considered proficient).
ND			X	Test data for all students and all grades will be aggregated and used to identify schools in need of improvement.
OH			X	SWD are included in assessment.
OK			X	SWD are included in assessment.
OR	X			Challenge-down assessments included in participation component.
PA	X			Document states that schools are concerned that the increased number of students with disabilities participating in the test will affect the funding they receive.
SC	X			Off-level results are factored into rating.
TN	X	X		Scores of students with disabilities are included for assessment of systems. Scores of students with disabilities are not included in value-added classroom evaluations.
TX	X			Scores of students with disabilities will be included in the 2002-2003 ratings. Alternate assessments will be included in the future.
UT			X	SWD included in assessment, but unclear how they are included in Title I AYP.
VA	X			All who take the SOL tests (which is defined to include students with disabilities) are included.
WY	X			All students, including students with disabilities, will be included in the accountability system (although WY’s accountability system did not appear to meet the criteria we used for a statewide accountability system).

Table 10. Communication with State Education Agency about Inclusion of Students with Disabilities in Accountability Systems

State	Current Inclusion	Future Inclusion	Other
AL	Included for Title I accountability in 2001-02	Included for overall accountability in 2002-03
AZ		It is anticipated that accommodated, modified, and alternate assessment results will be included in the new system.
CO	All students with disabilities are accounted for, either in general or alternate assessments. No child is excluded from accountability system.
CT	SWD are included.
HI		SWD will be included.
ID		Regular, accommodated, modified, and alternate assessments will be included.
IN			Formulas have not yet been set.
NJ		Working on inclusion, with the intent to include all SWD.
NC	SWD taking the regular test are included in School Performance Index, and annual high school cohort. This includes regular, accommodated, and modified administrations.
PA	Regular and accommodated administrations are included.
RI	Regular and accommodated administrations, as well as alternate assessments are included.
SD	Standard administrations (including standard accommodations) are included.	Non-standard administrations will be included in the future.
WA	Regular and accommodated test administrations are included.	There are plans to include alternate assessments.
WV	Currently, accommodated and regular assessments are included.	All SWD will be included in the future.
WY			All students with disabilities are included in determining Title I AYP. Alternate assessments are only included as a participation rate.

Discussion

The purpose of this report was to identify and describe various characteristics of state accountability systems that provide rewards, sanctions, or assistance to districts or schools on the basis of student achievement. One important feature of these systems that we were interested in was the degree to which students with disabilities are clearly included in the accountability determination.

Based on examination of state educational agency Web sites, as well as communication with accountability and Title I contact persons at state educational agencies, we obtained information on the current status, relevant indicators, growth components, Title I AYP definitions, and the inclusion of students with disabilities in state accountability and Title I accountability systems. States are at very different stages of school/district accountability system implementation. Because state accountability policies are frequently changing, especially as a result of recent federal legislation, the information provided in this report represents only what can be considered a "snapshot in time" of what states are doing or planning to do in the near future.

In addition to the difficulties associated with studying a "moving target," it was also often difficult to determine exactly which indicators truly counted in the determination of school accountability labels and consequences. As indicated in a previous report (Krentz, Thurlow, & Callendar, 2000), many states have a variety of indicators that are used for reporting purposes that may or may not be used for accountability purposes, as defined in this report. In some situations, consequences for schools were based primarily on one or two indicators, with the option for examination of other indicators if schools did not meet the initial requirements. Based on our analysis, student assessment scores and assessment score growth are the most commonly used indicators in these accountability systems. Other influential indicators include attendance, dropout rates, graduation rates, participation rates, and indicators of school safety. When determining Title I AYP status, many states primarily examine an increase in percent of students meeting proficient levels on state achievement tests, with several incorporating this calculation in a school index that may or may not include additional indicators.

States have incorporated a growth component into their accountability systems in a variety of ways. Many states include both a status and growth component such that lower performing schools must show more progress than higher achieving schools. In this way, states intend to help "close the achievement gap." However, growth is measured in a variety of different ways across states. Because comparing different cohorts of students across time can increase the volatility of school results (Linn, Baker, & Betebenner, 2002), it is often argued that schools should take this into account when they are making accountability decisions for schools. Several states are now incorporating a system that either tracks individual student progress over time, uses a rolling average of school scores, or allows schools to miss progress requirements on a small fraction of their indicators and still meet progress requirements for a given year. All of these options have been considered improvements (e.g., result in less volatile results) over other measures of school progress (Linn et al., 2002).

The degree to which students with disabilities are included in these accountability systems is also not always clear. In many situations, it may be the case that students with disabilities are included in the accountability system, but the information available in publicly available documents does not clearly articulate that these students are included. In a survey of state directors of special education, 15 respondents indicated that all students with disabilities were included in all components of the accountability system (Thompson & Thurlow, 2001). This represents information contrary to our findings, and is likely due to several differences in how information was collected in these two studies. For instance, "accountability system" was more stringently defined in the current study. Also, the current study relied on publicly available information linking the participation of students with disabilities to accountability calculations, whereas the earlier study simply required communication by the respondents.

In general, it is important for states to make clear who is included in determining accountability rewards, sanctions, and assistance. Otherwise, the public cannot adequately understand what it means for a particular school to be "superior," "adequate," or "underperforming." Furthermore, unclear articulation of which students are actually included in the accountability system undermines the promotion of high standards for all students. Ambiguity with regard to whether students with disabilities are included leaves one questioning whether they may actually be excluded from school and district accountability, as determined by the state.

In comparison to a similar previous analysis of state accountability systems (Krentz et al., 2000), it appears that there has been a decrease in the number of states that indicate students with disabilities are excluded from accountability systems. However, the number of states indicating that all students with disabilities truly count in the determination of school consequences remains quite low. Many state policies remain unclear in terms of which students with disabilities are included in accountability determination. It was also apparent from the current analysis that special schools, often serving students with special needs, are frequently held to different requirements for accountability purposes. It will be important to track these alternative accountability systems in order to ensure that they are not alienating students from the promotion of high standards present in other schools.

With many changes in elected representation expected in state houses, gubernatorial elections, as well as at the national level due to the 2002 elections, it will be difficult to predict the status of accountability legislation just a few short months in the future. It is likely that much of what is recorded here will change within the coming months. As states continue to revise and develop their accountability systems, it is anticipated that they will continue to provide the public with detailed information regarding the components of their accountability systems. In addition, it is anticipated that in revising state accountability policies to meet new federal requirements, states will move toward more appropriate inclusion of all students in accountability systems for schools and districts, ensuring that all students, regardless of disability status, are provided a learning environment with high expectations.

References

Center on Education Policy (2002). State high school exit exams: A baseline report. Washington, DC: Author.

Guy, B., Shin, H., Lee, S. Y., & Thurlow M. L. (1999). State graduation requirements for students with and without disabilities (Technical Report No. 24). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Johnson, D. R., & Thurlow, M. L. (2002). National study on high school graduation requirements and diploma options for youth with disabilities. Minneapolis, MN: University of Minnesota, Institute on Community Integration.

Krentz, J., Thurlow, M., & Callender, S. (2000). Accountability systems and counting students with disabilities (Technical Report 29). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Linn, R. (2000). Assessments and accountability. Educational Researcher, 29 (2), 4-14.

Linn, R.L., Baker, E.L., & Betebenner, D.W. (2002). Accountability systems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31 (6), 3-16.

Quenemoen, R. F., Lehr, C. A., Thurlow, M. L., Thompson, S. J., & Bolt, S. (2000). Social promotion and students with disabilities: Issues and challenges in developing state policies (Synthesis Report No. 34). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thompson, S., & Thurlow, M. (2001). 2001 State special education outcomes: A report on state activities at the beginning of a new decade. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M.L. (2000). Standards-based reform and students with disabilities: Reflections on a decade of change. Focus on Exceptional Children, 33(3), 1-16.

Wenning, R.J., Herdman, P.A., & Smith, N. (2002). No Child Left Behind: Who is included in new federal accountability requirements? Paper commissioned for a conference sponsored by the Thomas B. Fordham Foundation (February, 2002). Retrieved on September 12, 2002 from http://www.edexcellence.net/NCLBconference/Wenning%20Herdman%20and %20Smith.pdf

Appendix A: Inclusion Policies for Students with Disabilities in Accountability Systems among Four States Representing Full Inclusion Practices

Delaware

(http://www.doe.state.de.us/aab/DSTP_School_Accountability.html)

"Except for students who participate in out-of-level testing, students who test with non-aggregable conditions as defined in the Department of Education’s Guidelines for the Inclusion of Students with Disabilities and Students with Limited English Proficiency shall have her/his performance level included in the School Composite Score."

"For school accountability purposes, a student who tests but does not meet attemptedness rules as defined in the Department of Education’s Scoring Specifications, who participates in out-of-level testing or otherwise receives an invalid score shall be assigned to Performance Level 1."

"A student participating in alternate assessments shall have her/his performance level included in the School Composite Score."

Kentucky

(http://www.kde.state.ky.us/oaa/implement/Inclusion/Inclusions%20Document.pdf)

"Inclusion of Special Populations in the State-Required Assessment and Accountability Programs (703 KAR 5:070)" provides a detailed description of how students with disabilities participate in assessment and accountability systems.

(http://www.kde.state.ky/us/oaa/implement/DAC_Guide_2001-02/table_of_contents_2001.asp)

"Scores from alternate portfolios shall be included in the academic indices so that the data from an alternate portfolio completed by a student eligible to participate with an alternate portfolio contributes the same weight to the academic component of the accountability index as would the data for a student participating in the regular components of the assessment program at elementary, middle, or high school levels."

"The intent of the Kentucky General Assembly is that all students be included in the assessment and accountability components of KRS 158.645-KRS 158.6455. The only exceptions are those few students who cannot complete either the regular or alternate assessment components even with allowable accommodations, modifications or both due to medical or mental health conditions."

South Carolina

(http://www.state.sc.us/eoc/2001accountaanualjune1.doc)

"1. Students with accommodated administrations will be treated identically to students taking PACT in its standard form in absolute school and district ratings; . . .3. Students taking modified assessments, including "off-level tests", will be factored into the absolute rating according to the test score earned."

"Following their third administration, student assessment results from the PACT-Alternate assessment will be included in the calculation of the district but not the school ratings."

Vermont

(http://www.state.vt.us/educ/cses/alt/paric_guide_2001.htm)

"Some students will participate in the accountability system using one of three alternate assessment options (e.g., Modified Assessments, Adapted Assessments, or Lifeskills Assessments). Results from assessments using approved accommodations will be included in each school’s accountability index using the same point values that apply to standard administrations. Alternate assessment results will be entered into the accountability index using pre-determined score transformation rules. Invalid assessments will be designated as demonstrating ‘little or no evidence of achievement’ and will be assigned an accountability index value of zero."

Top of page

	Assess Scores	Assess Score Growth	Attend	AP Info	ACT/ SAT	Dropout Rate	Enrlmnt	Grad Rate	Retention Rate	Student/ Teacher Ratio	Suspension Rate	Staff Indctrs	Assess Partic Rate	Use of Resource Indctrs	Safety	Other indicators
AL	X	X														Other indicators will be added in the future according to state contact person
AK	X	X				X		X					X			Post-grad success, Local assessments
AZ	X	X				X		X								School site visit for schools labeled under performing two consecutive years
AR	X	X	X			X^a		X^a				X			X	Other school selected indicators, school narrative
CA	X	X	X^b					X^b
CO^c	X	X													X	Compliance with Safe Schools Act and the Colorado Literacy Act, having an educational improvement plan, evidence of reporting other information
CT	X	X
DE^d	X	X											X
FL	X	X											X
GA	X	X			X
HI	X	X	X^e					X	X							Other indicators may be added in future, according to state contact person
ID	X	X	X			X		X	X	X	X	X			X
IL	X	X
IN	X	X		X				X								Percent of graduating students who earn the academic honors diploma and Core 40 diploma.
IA	Accountability system did not meet criteria for inclusion in this report.
KS	X	X	X			X		X					X^b		X	Mastery of algebra, passing advanced math and science courses, local assessments
KY	X	X	X			X			X							Successful transition (for high school)
LA	X	X	X			X
ME^f	X	X														Completion of portfolio assessments, performance on local measures
MD	X	X	X			X
MA	X	X														Other indicators (dropout rates, attendance, improvement trends) may be used to determine whether full review of underperforming school is necessary.
MI	X	X														Indicators of instructional quality, indicators of learning opportunities, indicators of engagement
MN^f	X	X
MS	X	X
MO^g	X	X	X		X	X				X		X	X	X	X	Student/Administrator Ratio, Advanced Courses, Vocational Courses, Vocational Education Follow-Up, College Placement, GPA, Climate, Instruction, Preschool, Parent Communication, Community Involvement, Health Services, Foods Program, Transportation
MT	Accountability system did not meet criteria for inclusion in this report.
NE	Accountability system did not meet criteria for inclusion in this report.
NV	X	X^e	X										X
NH	Accountability system did not meet criteria for inclusion in this report.
NJ^h	X	X	X			X	X					X			X	Provide board approved program of guidance and counseling services Status of mandated programs/services, budget, external audits, transportation, health, other reports, substance abuse program, offering of other programs
NM	X	X	X			X								X	X	Parent/Community Involvement, Compliance with State Board of Education requirements, federal regulations, and state statutes, progress toward the EPSSgoals
NY	X	X				X										Certain exams, such as SATs and AP exams, may be used as approved alternatives to Regents High School examination scores.
NC	X	X				X							X			College University Prep/College Tech Prep Component
ND^f	X	X
OH	X	X	X					X
OK	X		X			X	X	X		X	X	X	X			Vocational-technical program participation and completion rates, college entrance and preparatory test-taking rates, parent involvement rates, test results in the context of socioeconomic status and finances of the school district, other indicators reported through Oklahoma Education Indicators Program
OR	X	X	X			X							X			Local data used for k-2 AYP
PA	X	X	X				Xⁱ	X^e					X			Job-related placement for Vocational-Technical Schools. Student technology literacy^e, percent of migrant students enrolled in “schools in need of improvement”^e, percent of LEP students who have obtained English proficiency^e
RI	X	X	X^b			X^b		X^b			X^b	X^b	X^j		X^b	(Safety is part of school climate and school work climate.)
SC^k	X	X	X		X	X			X^e	X		X	X^j			Parent Involvement, External accreditation, Enrollment, placement, and graduation rates for Career and Technology centers, Other factors promoting or maintaining high levels of achievement and performance, criteria appropriate to each school’s mission (used for special schools), School Readiness^e
SD	X	X^e														More indicators may still be added.
TN	X	X			X	X
TX	X	X^l	X^l	X^l	X^l	X										Advanced course completionⁿ
UT^f	X	X
VT	X	X											X			Local assessments
VA	X	X													X	Must also meet compliance with promotion/retention policies, course offerings, and staffing regulations.
WA	X	X
WV^m	X	X	X			X										Random audits are also completed to check compliance with 97 standards.
WI	X	X											X
WY^f	X	X
	46	45	20	2	5	20	3	12	4	4	3	7	14	2	9