This report is the sixth analysis of state reports conducted by the National Center on Educational Outcomes (NCEO) to examine the extent to which states publicly report information about students with disabilities in statewide assessments. We present descriptions of statewide testing systems and examine whether these systems included participation and performance information for students with disabilities, as indicated by publicly available data. The majority of our information was obtained by analyzing states’ Department of Education Web sites. If disaggregated information was not posted, the states were then asked to submit public documents that included these results.

For the 2001-2002 school year, the number of states that reported both participation and performance data on students with disabilities for their general assessments was 35. This number was an increase from the 2000-2001 school year, in which only 28 states reported both participation and performance data. For the 2001-2002 year, participation data were presented in a variety of ways. The most common way was to present the number of students tested. Almost all states that reported participation data did this. Twenty-two states went beyond the numbers to report rates of participation.

General assessment performance data for students with disabilities also were reported in a variety of ways by states. Performance data reported on state’s general assessments, more often now than in previous years, compared students with disabilities to general education students or the total population of students. The results clearly illustrate the achievement gap that exists between these two groups. Still, the gap does vary dramatically across states.

Alternate assessment participation and performance reporting for 2001-2002 was available in 22 states. This was up considerably from only 13 states in 2000-2001. Other states provided only participation data (5 states) or only performance data (5 states). The nature of the data presented on alternate assessments was usually just an overall count of students participating or an overall rate of students passing. Though some states did break participation and performance information down by grade level or content matter, many states still only provide aggregated numbers.

Although not all states were yet reporting on the participation and performance of students with disabilities on either the general assessment or the alternate assessments administered in 2001-2002, the number doing so continued to increase from previous years. In addition, the nature of the reporting seemed to have improved. Nevertheless, there are many reasons to still be disappointed. Chief among these is the fact that not all states report. This was a request of IDEA 1997, the law that was enacted a full three years prior to this data reporting. Based on the states that do report, however, it is possible to generate some recommendations for where and how to move things forward. Several ideas are presented in this report.

Overview

Since 1997, the National Center on Educational Outcomes has been collecting data on the inclusion of students with disabilities in statewide assessment systems. This effort was initiated because of the recognition that excluding large numbers of students with disabilities from state assessments presents an inaccurate picture of how all students are performing (Thurlow, House, Boys, Scott, & Ysseldyke, 2000; Thurlow, Ysseldyke, Erickson, & Elliott, 1997; Zlatos, 1994). It is impossible to get a sense of the effectiveness of public education if all participants are not being measured. In addition, if the performance of students with disabilities is not assessed, teachers and administrators feel less pressure to ensure that this group of students is making visible academic progress (Elliott, Thurlow, & Ysseldyke, 1996).

Cibulka and Derlin (1995) proposed several purposes that public reporting of student assessment data serves. First, results can be used to make educated decisions about educational programs and general school effectiveness. Second, it is possible to evaluate the achievement of students. An effective accountability system is one that shares results both with educators and with the general public. When shared with others, data should be clear and accessible so that all stakeholders can use the data to reach helpful conclusions and determine whether school programs are working effectively.

Until the mid-1990s, legislation did not address the issue of students with disabilities being excluded in significant numbers from large-scale assessments. However, in 1994, the Elementary and Secondary Education Act (ESEA) amended Title 1, which required that states assess all students including students with disabilities using state tests to determine their progress toward state standards. Title 1 also required that disaggregated performance information be reported in a public report of school progress (U.S. Department of Education, 1999). Following upon ESEA, in 1997 the Individuals with Disabilities Education Act (IDEA) of 1990 was amended to require that states report state and district-wide assessment information for students with disabilities with the same frequency and in the same detail they report for students without disabilities. Both participation and performance information must be reported for both the general assessments and alternate assessments (National Research Council, 1999). States were required to develop alternate assessments to assess the performance of students who were unable to participate in the general assessment. Students participating in alternate assessments most often were students with significant cognitive disabilities.

The 2001 reauthorization of the Elementary and Secondary Education Act, the No Child Left Behind (NCLB) Act, required that by the beginning of 2002-2003 school year, assessment results would be reported at the classroom, district, and state levels (Fast, Blank, Potts, & Williams, 2002). Results also had to be publicly reported by the start of the following school year (i.e., Fall 2003). This means that most states need to speed up the process of scoring, analyzing, and disseminating results to students, schools, and districts. This has undoubtedly led to more Web-based reporting practices, which are faster and less expensive than printing lengthy reports.

Educational policies over the past decade have become more directive about participation in assessments. In the early 1990s, there was great variation in state guidelines that addressed the participation of students with disabilities in state assessments (Thurlow, Ysseldyke, & Silverstein, 1995). Participation rates of students with disabilities were known to be quite low and to vary widely from one state to another (Erickson, Thurlow, & Thor, 1995; McGrew, Thurlow, Shriner, & Spiegel, 1992; Shriner & Thurlow, 1992). In 1992, only 19 states reported assessment information, and of those 19 states, most reported participation rates of less than 10% of students with disabilities (Shriner & Thurlow, 1992). Many states reported that they did not know how many of their students with disabilities participated. If more than 10% of the population is not being assessed, it is impossible to get an accurate picture of how all students are performing (Thurlow & Thompson, 1999). In examining assessment data from the 2000-2001 school year, we found 38 states reported participation information on at least some of their state assessments (Thurlow, Wiley, & Bielinski, 2003). According to No Child Left Behind, in order to be considered to be making adequate yearly progress, schools must have 95% of their students participate in their state assessment. This requirement is broken down even further into subgroups, with the requirement that 95% of each subgroup must be tested. Recent proposed flexibility proposals have allowed that the 95% can be achieved through an averaging process across two or three years. This “flexibility” does not allow for wide variations in participation, however. To average 95%, states have to be very close to that percentage in each year that figures into the average.

Within the past several years, a growing number of states have begun publicly reporting information about the participation and performance of students with disabilities in their statewide assessment system (Bielinski, Thurlow, Callendar, & Bolt, 2001; Thurlow, Langenfeld, Nelson, Shin, & Coleman, 1998; Thurlow, et al., 2003; Ysseldyke, Thurlow, Langenfeld, Nelson, Teelucksingh, & Seyfarth, 1998). During the 2000-2001 academic year, 76% of states reported both participation and performance for students with disabilities on at least some of their state tests (Thurlow et al., 2003). When considering having either participation or performance, this percentage increases to 88%. Clearly states are now beginning to report these disaggregated data, though only 56% of states report this information for all of their state assessments (Robelen, 2004).

Method

Because most states now report assessment results on the Web, we began our search for information by reviewing every state’s Department of Education Web site. We began collecting data in May 2003 and collected information for the 2001-2002 school year. We recorded assessments administered and documented whether participation and performance information was reported for students with disabilities. We also examined the way in which participation was reported and whether participation and performance information was given for students who took the test with accommodations. By May 2003, a large percentage of the states had already posted their 2001-2002 assessment data on-line in a way that made them easy to locate and understand.

On June 5, 2003 we mailed a letter to each state Director of Assessment (see Appendix A) outlining our findings from the state’s Web site. We asked them to review our findings, correct for any misinformation, and provide the public document or Web site at which the correct information was available. We asked that they send us these changes by June 27, 2003. Many states that had changes to make either sent us printed documents with the data or directed us to a Web page that we had not found in our search. Several states gave us dates by which time their disaggregated assessment results should be posted. Overall, we received responses from 20 Directors of Assessment.

In order to ensure that our findings were as accurate as possible, we followed up these efforts with a letter to each state’s Director of Special Education (see Appendix B). These letters were mailed on August 5, 2003. The letters asked the Directors to review our findings and make any changes by August 29, 2003. For states from which we had already received a response from the Director of Assessment, we noted that in the letter by stating that “These results were verified by your state’s Director of Assessment, but if you have anything to add, please let us know.” For states from which we did not hear from the Director of Assessment, we sent the same letter to the Director of Special Education as we had sent to the Director of Assessment. Of the 50 states to which we sent letters, 25 responded with either corrections or to verify that the information that we had was correct.

Characteristics of State Assessment Systems

Appendix C lists all the state mandated general assessments that we identified for the 50 states. This list includes the state, the name of the test, the grades and content areas tested, and whether the state had publicly available disaggregated participation and performance data for students with disabilities for their 2001-2002 state assessments. We identified 111 separate statewide tests or testing systems. Thirty-five states had more than one general assessment.

Figure 1 breaks down the 111 testing systems by type: norm-referenced tests (NRT), criterion-referenced tests (CRT), exit tests used as a gate for graduation or earning a particular type of diploma (EXIT), and tests that combined standardized NRTs with additional state-developed test items (NRT/CRT). While we recognize that many exit exams may also be NRTs, CRTs, or both, the high stakes consequences for students on these exit exams indicated a need to create a separate category for these tests.

Figure 1. Types of General Assessments

Criterion-referenced tests (CRTs) comprise 52% of all the assessments that states administer. In fact, only four states (Hawaii, Iowa, Montana, South Dakota) do not administer a CRT and an additional four states (Delaware, Florida, Indiana, Missouri) do not have an separate CRT, but rather have added criterion-referenced items to their NRT (CRT/NRT). The next largest number of tests administered are norm-referenced tests (NRTs), which comprise 22% of all assessments, followed closely by exit exams (21%). These numbers are similar to the 2000-2001 assessment pattern, in which 51% of tests were CRTs, 33% were NRTs, and 22% were exit exams (Thurlow, et al., 2003).

States Reporting Disaggregated 2001-2002 General Assessment Data for Students with Disabilities

Figure 2 summarizes the different ways in which general assessment data were reported for all 50 states. Seventy percent of states reported disaggregated participation and performance information on students with disabilities for all their assessments, 8 percent reported performance for all assessments (but not participation data), 18 percent reported participation and performance information for some assessments, and 4 percent did not report any disaggregated information.

Figure 2. States that Disaggregate Assessment Results for Students with Disabilities
 

Figure 3 indicates which of the 50 states: (1) reported participation and performance for all of their general state assessments (70%), (2) reported performance results on all general assessments, but not participation data (8%), (3) reported participation and performance for some of their general assessments (18%), and (4) did not report either participation or performance results for any of their general assessments (4%). States that reported disaggregated data for students with disabilities at the state level generally reported results at the district and school levels, too.

Thirty-five states reported participation and performance results for students with disabilities on all of their tests. As evident in Figure 3, these states are spread across the U.S; they are states with both small and large populations. The states that reported disaggregated 2001-2002 data for their general assessments did so regardless of whether they had just one assessment or multiple assessments (20 of the 35 had more than one assessment), and regardless of whether they tested in just a few grades or in as many as 10 grades.

Figure 3. States that Report 2001-2002 Disaggregated Results for Students with Disabilities

Of the 9 states that reported participation and performance information for some of their assessments, the majority (n=7) were only missing data on one test. These states were Florida, Louisiana, South Dakota, Texas, Washington, West Virginia, and Wyoming. Virginia was missing performance data for only one assessment, though participation data were missing for two assessments. It is important to note that all states that reported disaggregated performance data did so at the state level except Wyoming, which only provided disaggregated information at the district level.

States Reporting 2001-2002 Alternate Assessment Data for Students with Disabilities

As shown in Figure 4, results from our Web searches and mailings revealed that 22 states publicly reported both participation and performance results for their alternate assessment. An additional 5 states reported performance only, and 5 states reported participation only. Thus, 36 percent of states did not report any type of information about their alternate assessment. However, 44% of states did report both participation and performance for their alternate assessment.

Figure 4. Information States Reported for their Alternate Assessment

Figure 5 illustrates which states reported alternate assessment participation and performance data. There is no obvious geographic pattern to the states that did not report alternate assessment data. The states with no information are not states that did not have an alternate assessment in 2001-2002. According to Thompson and Thurlow (2001), all but 2 states had an alternate assessment approach by 2001 and all but 16 states had decided how scores from the alternate assessment would be reported. The no information states most likely are those 16 states plus some additional states.

Figure 5. States Publicly Reporting Data for the 2001-2002 Alternate Assessment

Assessment Participation in 2001-2002

General Assessment Participation Results

Among the states identified as providing participation data for students with disabilities, the way in which this information was reported varied (see Appendix D). Figure 6 illustrates the number of assessments with disaggregated participation data and how those participation data were reported. Information is presented in terms of the number of assessments for which participation data were available, not in terms of the number of states. For example, in Alabama there are three assessments and each is counted separately. We used this approach because not all states report participation in the same way across assessments. For example, one state might report only a count of students tested for one assessment, but for another assessment it might report a count tested, a percent tested, and a percent not tested.

Figure 6. Participation Reporting Approaches for General Assessments (Number of Tests = 86)

Of the 86 general assessments that reported participation data, only 15 states reported both a count and a percentage of students tested; sixty-two assessments had just a count of students tested. Overall, states reported the percent of students tested for 22 assessments; 14 also provided a count as well as a percentage. For 15 assessments, the number or percentage of students who were not tested was provided, and for 8 assessments, states provided information about the number or percent of students who were exempted or excluded. For 6 assessments, states provided information about the number or percent of students who were absent.

Figure 7 illustrates the participation rates reported in those states for which there was clear participation rate information reported. Though the percentage of students tested was given for 22 assessments, those assessments came from only 15 states; an additional 4 states gave the percent of students not tested. While it may have been possible to calculate participation rates for other states as well, using information that was reported about student enrollment and the number of students tested, we did not take the extra step to do the math calculations. This is because we were concerned about the information that was readily available. However, if the state did provide only the percentage of students not tested, we did report the percentage of students tested in the table. It is important that states report the percentage of students tested, in addition to just a count, because this presents a more accurate picture of how many students are participating. These rates should ideally be based on the school enrollment on the day of testing (Ysseldyke, et al., 1998); however, using the December 1st Child Count data is also an acceptable option if test day enrollment is not available.

Figure 7. Percentages of Students with Disabilities Participating in General Assessments in Those States with Clear Participation Reporting of Rates

To summarize participation rate information, we selected one grade to portray in Figure 7. In most states, participation in the middle school/junior high school math test was used. If the state tested in more than one grade in the middle school level, the 8th grade test data were used. Appendix E contains information about the tests and exact grades used for Figure 7. Not all states provided data broken down in this way. For example, in Nevada, the grade 8 data are combined for the reading and math test, and the South Carolina data are for all grades combined for the math assessment. Oklahoma, Texas, and West Virginia also provided a rate, but as it was for all subjects and grades, it was not included. Three other states (Illinois, Kentucky, New Hampshire) provided a participation rate, which was the number of students with disabilities who participated out of all students rather than a percent of students with disabilities who were tested. Though this is more helpful than not providing a rate, it was not included in Figure 7 because it was a different type of rate. (Maine provided a rate for the number of students with disabilities who took the test with accommodations, but not for the number of students with disabilities who tested without an accommodation; therefore, those results are not included.) It is important to note that results in Figure 7 were obtained from different types of tests that were being used in these states. Nevertheless, during this 2001-2002 academic year, participation rates ranged from 71.1% to 99.1%. Five out of the 12 states had participation rates of 95% or higher.

Alternate Assessment Participation Results

Figure 8 illustrates how different states reported participation for their alternate assessment. Appendix F outlines in more detail all the ways that information is reported. Twenty-seven states provided participation information for their alternate assessment. Fourteen states provided only a count and not a percentage of students tested or not tested. Three states provided the opposite: the percentage of students tested or not tested, but not the number of students tested or not tested. Ten states gave both a count and the percentage of students either tested or not tested.


Figure 8. Participation Reporting Approaches for Alternate Assessments (Number of Tests = 27)

Other Information Collected for 2001-2002

In our analysis of state reporting for 2001-2002, we looked at additional characteristics of states’ information. Specifically, we looked at information available on accommodations used, and if available, performance when accommodations were used. We also examined the quality of Web-based reporting.

Accommodations

Fourteen states provided information about students who took an assessment with an accommodation. In some cases, states reported on standard accommodations (those considered appropriate and not ones that change the constructs measured by the assessment); in other cases they reported on nonstandard accommodations (which generally were considered to change the constructs measured – and might be referred to as “non-allowed” – although IEP teams could select them), and in other cases they reported on both or did not specify which.

Table 1 describes the information the 14 states provided. Appendix G contains additional information about the information provided by the 14 states, with details about the participation and performance of students in each category that the state provides. Five states broke down student participation and performance by accommodation (e.g., directions read orally, Braille, extended time), whereas 9 states provided only overall information on students who, in general, used accommodations.

Table 1. States that Reported Information about Accommodations

 Note: SWD= Students with Disabilities

Quality Analysis of Web-based Reporting

After examining every state’s Department of Education Web site, it became evident that some states presented data in a much more accessible format than others. Because assessment data are reported on the Web in most states, it is crucial that these data be clear and easy to access. We decided to collect data for each state that reported results for students with disabilities online and examine the quality of the reporting on the Web site. It is important to note, however, that because Web sites are frequently updated, it is possible that some of our findings no longer hold true.

To describe what we found, we identified the elements that we would like to see on a Web site. The following eight elements were evaluated (see Appendix H):

1. The word(s) on which one must click on the Department of Education’s homepage.

2. How many steps, or “clicks,” it took to get from the state’s homepage to the disaggregated results.

3. The proximity of special education data to general education/all students data.

4. The proximity of special education data to the Alternate Assessment data.

5. Whether the term “proficient” was defined for student performance.

6. Whether the data all appear on one page when printed, or if some of the data are cut-off.

7. Whether the date of testing appeared on the same page with the assessment results (giving just the year- e.g., 2001, did not count because that did not indicate the academic year from which the data came).

8. Whether there were at least two years of trend data available on the same page or a direct link given on the page (trend is 2000-2001 data and 2001-2002 data).

When finding disaggregated data, one must click on a word on each state’s Department of Education homepage to begin looking for the data. The clarity of each initial word/phrase varied widely by state. Of the 85 assessments with disaggregated Web-based results, 17 used the word “assessment.” Ten used the word “test” or “testing.” Five more used the word “accountability.” Several other states gave the name of their state test as a first step to retrieving assessment data. These are relatively clear indicators of where to find test data. In contrast, some states used words that did not indicate assessment data, such as “More,” “Statistics,” “Administrators,” and “Programs and Services.” It is important for states to make assessment information as publicly accessible as possible so that the effort of posting state assessment results reflects an intent for the public to view and use the data.

To get from a Department of Education Homepage to disaggregated participation and performance data, it is often necessary to “click” several times and follow a series of links through the Web page. The number of clicks it took to get from the homepage to the actual data ranged from 1 to 7 clicks. The most common number of clicks between the homepage and the disaggregated data was 3 clicks. Figure 9 shows the range of steps required to find assessment results on Web sites, starting from the homepage. The numbers reflect tests (total n=85), not states, for which we found disaggregated performance results. We found that the more clicks it took to reach the data, the easier it was to get lost along the way and the more difficult it was to find the results again at a later time.

We also examined the proximity of special education data to the data of general education students and the total population of students. States that had the most comprehensive presentation of data posted disaggregated results either with, or clearly linked to, results for all students. Figure 10 illustrates the range of distances that results for students with disabilities were from results for general education students or the total population of students. This figure is based on the total number of tests (n=85), not the number of states. As evident in the figure, over half of the general assessments had results for students with disabilities on the same page with results for general education students or the total population of students.

Figure 10. Proximity of Special Education Data to Data for All Students or General Education

Of all the assessments with Web-based disaggregated data, 69% had results posted for all students on the same page as results for students with disabilities. Another 15% of assessments had their special education results one click away from their general assessment results, and 11% had these data separated by two or more clicks. Though the majority of these assessments had the special education data 3 clicks away from data for general education students, one state had 2 assessments that were 8 clicks away. For 5% of the assessments, a paper document had been scanned onto the computer and disaggregated data were provided with data for all students in the same document, though on different pages. A problem with this, however, is that many documents are long, some over 300 pages. It takes a long time to look through the document to find those data for which you are searching, and it is quite easy to miss data, too.

We also examined the proximity of the special education performance results to performance or participation information for the state’s alternate assessment. Results varied widely. For 11 assessments, these data were on the same page and for 6 assessments the data were in the same document. The alternate assessment information was 1 click away for 2 assessments, 2 clicks away for 12 assessments, and 3 clicks away for 9 assessments. For the remaining 12 assessments, alternate assessment data were at least 4 or more clicks away, and for three of those, the data were 8 clicks apart.

Another quality issue that we assessed was whether there was a definition for proficiency where performance results were presented. Many people look at assessment data to see the percentage of proficient students in a certain grade, subject, or subgroup. It is necessary that states indicate either by a written term (e.g., basic, proficient, advanced) or by defining the categories (e.g., if numbers are used, a key for what each number stands for should be provided). Of the 85 assessments in our analysis, 65 provided a definition of proficiency. Many states used the performance levels of Below Basic, Basic, Proficient, and Advanced. However, others used just the percent passing or meeting standard versus percent not passing or not meeting standard distinction.

Two other quality issues that we explored were whether the data were able to be printed on one page and whether the date of the assessment was provided with the results. In several cases we tried printing the data, but what appeared on the Web site did not all print onto a page, even when printing using a horizontal page layout. Frequently data on the right side of the page was cut off. For 19 of the 85 assessments (22%), the data that appeared on the Web site did not all print. The second additional indicator reflected a belief that it was important to have the date of testing on the same page as the test results so that people would know the year of the data they were examining. To be considered as having the date, we did not count states that only gave one year (e.g., 2002) because it was not clear whether that data came from the 2001-2002 school year or the 2002-2003 school year. States needed to give an academic year (e.g., 2001-2002) or provide the month or season (e.g., fall 2001) in which the assessment occurred. Twenty-two assessments (26%) did not provide the date with their assessment data.

A final quality issue we chose to examine was whether two years of assessment trend data were reported. Of the 85 assessments, 36 (42%) did provide at least two years of trend data (2001-2002 and 2000-2001). We only counted states that had two years of data available together on the same page or one link apart. While we recognize that more assessments might have data from the previous year available on the Web site, we believed that it was too difficult to examine trends and patterns when the data were spread out.

States such as Nevada and Washington presented particularly clear and thorough results. Disaggregated assessment results were posted only three clicks from the homepage. For Nevada, results for students with disabilities were given on the same page as results for all students, and in Washington, results were only one click apart. Though Nevada did not provide results for its alternate assessment, participation for Washington’s alternate assessment was given on the same page as results for both all students and students with disabilities; both states defined “proficiency.” The results for Nevada and Washington had the date of testing on them and were able to be printed without any part being cut off. Though Nevada did not provide trend data, Washington had data from the previous three years of testing just one click away.

Assessment Performance in 2001-2002

General Assessment Performance Results

We examined the performance of all students, and then the performance of students with disabilities. When examining performance across states, it is important to remember that the scores from each state are based on different tests. These tests may emphasize different standards and are likely to differ in difficulty. In addition, there is great variability across states in terms of the percentages of students with disabilities who have been included in the assessments. Thus, it is not appropriate to compare performance across states. It is possible, however, to examine the performance differences within each state between all students and students with disabilities.

Performance results are reported for both reading and math assessments because these content domains are the ones assessed by most states and are the content areas required first by NCLB to be assessed, reported, and included in accountability. For greater comparability in what we report and because states are now moving away from norm-referenced tests toward a wider use of criterion-referenced tests, we only report performance on CRTs. We also report performance on Exit exams that students are required to pass to graduate from high school with a standard diploma.

We separated grade levels into three categories: elementary (3-5), middle school (6-8), and high school (9-12). For our summary, we chose to present only one grade for each level. When available, 4th grade was used to represent the elementary level, 8th grade to represent the middle school level, and 10th grade to represent the high school level. These grades were chosen because they are the grades at which the greatest number of states test students. If data from those grades were not available, the grade below was used, followed by remaining grade if no other data were available. The number in the parenthesis next to the state’s name indicates the grade from which the data were obtained. Appendix I reports the name of the test we used and the grade.

Although most states reported the performance of all students and then the performance of subgroups, such as students with disabilities, some states did not report the performance of all students. When these data were not available, the performance of general education students was given. Because the performance of general education students as a group may be slightly higher than the performance of all students as a group, we have indicated those states with “all students” actually based only on general education students by an asterisk after the name of the state. These data are presented in Figures 11-18.

Figure 11. Elementary School Reading Performance on Criterion-Referenced Tests

Figure 12. Middle School Reading Performance on Criterion-Referenced Tests

Figure 13. High School Reading Performance on Criterion-Referenced Tests

As evident in Figures 11-13, the performance of students with disabilities in reading is generally much lower than the performance of all students. Though the gap is greater in some states than in others, students with disabilities are always performing below all students. As students move from elementary to high school, the gap grows wider. At the elementary level, the widest gap was 43.7 points in New Jersey. In middle school the greatest gap was 56 points in both New Jersey and Delaware. At the high school level, the largest gap was 67 points in Connecticut. Though these are the largest gaps, the pattern is the same for most states.

Performance of all students and students with disabilities on states’ 2001-2002 mathematics assessments is shown in Figures 14-16. The Figures cover elementary, middle, and high school in a manner similar to the reading figures and with the same cautions.
 

Figure 14. Elementary School Mathematics Performance on Criterion-Referenced Tests

Figure 15. Middle School Mathematics Performance on Criterion-Referenced Tests

Figure 16: High School Mathematics Performance on Criterion-Referenced Tests

As shown in Figures 14-16, the gap between students with disabilities and all students is quite similar to the gap found for reading assessments. The gap for math assessments exists in all states and varies considerably from state to state. The gap also increases by grade level. In elementary grades, the largest gap was 39.2 points in Delaware. In middle school, the largest gap was 51.2 points in New Jersey, and in high school it was 64.3 points in Connecticut.

Figures 17 and 18 show the results of high school reading and math exit exams. States administer exit exams in different grades. The number in the parenthesis next to the state’s name indicates the grade from which the data come. Only those states that report disaggregated results for students with disabilities are included in these figures. Also these results reflect only the first administration of the exit exam. States offer multiple retest opportunities for their exit exams and the percent passing increases with each retest. Often the gaps between general and special education students become very small on retesting.

Figure 17.  Percent Passing Minimum Competency/High School Reading Exit Exam

Figure 18. Percent Passing Minimum Competency/High School Mathematics Exit Exam

The figures presented here for first-time testing show that large gaps exist for exit exams, though the percent of students passing the exit exams varies widely by state. For both reading and math, New Jersey had the largest gap (52.3 points for reading; 50.9 points for math). The gap on reading tests was small for both Louisiana (13 point difference) and Maryland (11.7 point different) though the percent passing in Louisiana was very low and the percent passing in Maryland was quite high. For math, again the gap was smallest in Louisiana (16 point difference) and Maryland (17.6 point difference) and the percent passing in Louisiana was low whereas the percent passing in Maryland was relatively high.

Discussion

This sixth analysis of state education public reporting shows that states continue to make progress in the amount of information they report on the participation and performance of students with disabilities. Still, it is disappointing to see that not all states are reporting disaggregated information, and fewer than half of the states are reporting both participation and performance information for their alternate assessment.

A total of 48 states reported some information for their state assessments. Of these states, only 35 reported participation and performance for all of their assessments. An additional 9 states provided participation and performance information for some of their assessments, and 4 states reported performance data for all of their tests, though not participation. The number of states reporting both participation and performance rose from 28 states during the 2000-2001 academic year to 35 in 2001-2002. When examining participation rates for students with disabilities, the data, though variable, were much more stable than results from the 2000-2001 school year. In 2000-2001, the data ranged from 30% to 97.4% participating, whereas in 2001-2002 the data ranged from 71.1% to 99.1%.

When examining alternate assessments, only 32 states reported any information. Though this is an increase from only 25 states during the 2000-2001 year, states clearly are not reporting on their alternate assessments at the same level as they are for their general assessment. Twenty-two states provided both participation and performance data for their alternate (up from 13 states in 2000-2001), 5 states gave performance data only, and 5 states gave participation data only. The lower level of alternate assessment reporting seems to be due only in part to the fact that some states were still working on the development of their alternate assessments. Many of the non-reporting states had alternate assessments and even had reporting plans (Thompson and Thurlow, 2001), but had just not reported.

When examining ways that states report participation information for the general assessment, we found that for most general assessments (72%) only a count was reported. For 22 assessments (26%), states reported the percentage of students tested. While a rate is a more meaningful way to provide information about participation than just providing the number of students tested, only approximately ¼ of all assessment data included a rate. For 15 assessments (17%), states provided either the number or percentage of students who were not tested. For 6 assessments (7%), states gave either the number or percentage of students who were absent from testing.

When we examined the performance of students, we found that for the general assessment large gaps existed between students with disabilities and all students. Though some gaps were significantly larger than others, the gaps were noticeable for all states that provided performance data. Gaps increase as students get older.

This was the second year that we systematically examined that quality of Web-based reporting and how easily accessible the data were. Though many state Department of Education home-pages provided clear first links to their data by using words such as “Assessments,” “Student Testing,” or “Accountability,” other states still used vague terms such as “Statistics,” “Administrators,” and “Programs and Services.” The most common number of clicks it took to get to the disaggregated data from the homepage was three clicks. For 69% of the assessments, data for special education students and all students were presented on the same page. For 15% of the assessments, the data was one click apart. Quality issues were also examined such as whether all the data printed, whether the data had a date on it, and whether more than one year of data was presented so that trends could be examined. If districts, schools, teachers, parents, and others are going to use publicly reported data, it is essential that the data not only be reported but that they be easily accessible. There were no noticeable improvements in the accessibility of data from the 2000-2001 reports.

Recommendations for Reporting

With the push from No Child Left Behind to provide assessment data to schools by the start of the school year, Web-based reporting has clearly become the primary vehicle for sharing data with the public. It is crucial, then, that the data be easy to both locate and comprehend. Based on our analyses of both Web-based and paper reports, we make the following recommendations:

1. Report not only the number of students with disabilities assessed, but also the percentage assessed. When states provide the number of students assessed, this information is less helpful than when a percentage is provided. By giving a percent, people are able to get a more accurate picture of how many students are participating in the state assessment system.

2. Ensure that Web-based assessment information is dated so the viewer knows what testing year it was from and that it can be printed. When states only indicate the testing year by giving one year (e.g., 2002), it is unclear whether the data are from the fall or spring. States should either provide the entire school year, such as 2001-2002 or provide a month or season, such as Fall 2001. Another quality issue is ensuring that the test data all prints so that people can have a hard copy of the test information. We recommend that if the data is on the Web and will not all print onto a standard piece of paper, a “print-format” option should be provided, which can provide the data in a standard format that can be printed.

3. Report results for the alternate assessment. Though states are finally beginning to provide participation and performance data for their general assessment, they are still slow about reporting that information for their alternate assessment. This information should be provided so that the public can see how all students are performing.

4. Report the number and percent of students with disabilities using accommodations. Because of various disabilities, many students are not able to take the general assessment in the standard format, and thus are provided with accommodations. Many states consider the scores of some of these accommodated assessments to either not count or to count as “not-proficient” because they are non-standard accommodations. In some states, the number of students participating using non-standard accommodations is quite high. If these numbers are not reported, then the picture painted of how all students are doing will be inaccurate.

Although there has been improvement in the reporting of assessment participation and performance data for students with disabilities, there is still much more that can be done. To some extent, the greatest improvement is in the mere reporting of the data. This should have been accomplished several years ago. What is still needed in many states – not all, of course – is a serious commitment to ensuring that the reporting for students with disabilities is on a par with that of other students. In other words, it should be just as easy to find these data, and they should be just as clear as the data for general education students. Equality in presentation – easily accessed and transparent – should be the first criterion on which the reporting of data for students with disabilities is judged. While we are almost there, we are not there yet.

References

Bielinski, J., Thurlow, M.L., Callender, S., & Bolt, S. (2001). On the road to accountability: Reporting outcomes for students with disabilities (Technical Report 32). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/Technical32.htm

Chibulka, J.G. & Derlin, R.L. (1995). State educational performance reporting policies in the U.S.: Accountability’s many faces. (ERIC Abstract ED 401 613).

Elliott, J.L., Thurlow, M.L., & Ysseldyke, J.E. (1996). Assessment guidelines that maximize the participation of students with disabilities in large-scale assessments: Characteristics and considerations (Synthesis Report 25). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/synthesis25.html

Erickson, R.N., Thurlow, M.L., & Thor, L. (1995). 1994 state special education outcomes. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.
Fast, E.F., Blank, R.K., Potts, A., & Williams, A. (2002). A guide to effective accountability reporting. Washington, DC: Council of Chief State School Officers.

McGrew, K.S., Thurlow, M.L., Shriner, J.G., & Spiegel, A.N. (1992). Inclusion of students with disabilities in national and state data collection programs (Technical Report 2). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

National Research Council (1999). Testing, teaching, and learning: A guide for states and school districts. Committee on Title 1 Testing and Assessment, R. F. Elmore and R. Rothman, (Eds.) Board on Testing and Assessment, Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Robelen, E.W. (2004). States given more leeway on test rule. Education Week, 23(30), 1, 28-29.

Shriner, J.G., & Thurlow, M.L. (1992). State special education outcomes 1991. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thompson, S. J., & Thurlow, M. L. (2001). 2001 state special education outcomes: A report on state activities at the beginning of a new decade. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/2001StateReport.html

Thurlow, M.L., House, A., Boys, C., Scott, D., & Ysseldyke, J. (2000). State participation and accommodations policies for students with disabilities: 1999 update (Synthesis Report 33). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/Synthesis33.html

Thurlow, M.L., Langenfeld, K.L., Nelson, J.R., Shin, H., & Coleman, J.E. (1998). State accountability reports: What are states saying about students with disabilities? (Technical Report 20). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/Technical20.htm

Thurlow, M.L., & Thompson, S.J. (1999). District and state standards and assessments: Building an inclusive accountability system. Journal of Special Education Leadership, (12)3-10.

Thurlow, M., Wiley, H.I., & Bielinski, J. (2003). Going public: What 2000-2001 reports tell us about the performance of students with disabilities (Technical Report 35). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/Technical35.htm

Thurlow, M.L., Ysseldyke, J.E., Erickson, R.N., & Elliott, J.L. (1997). Increasing the participation of students with disabilities in state and district assessments (Policy Directions No. 6). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/Policy6.html

Thurlow, M.L., Ysseldyke, J.E., & Silverstein, B. (1995). Testing accommodations for students with disabilities. Journal of Remedial and Special Education, (16)260-270.
U.S. Department of Education. (1999). Peer reviewer guidance for evaluating evidence of final assessments under Title I of the Elementary and Secondary Education Act. Washington, DC: Author.

Ysseldyke, J.E., Thurlow, M.L., Langenfeld, K., Nelson, J.R., Teelucksingh, E., & Seyfarth, A. (1998). Educational results for students with disabilities: What do the data tell us? (Technical Report 23). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/TechnicalReport23/technical_report_23.htm

Zlatos, B. (1994). Don’t test, don’t tell: Is “academic red-shirting” skewing the way we rank our schools? The American School Board Journal, (181)24-28.

The National Center on Educational Outcomes is examining states’ public reports on 2001-2002 school year assessment results. We have reviewed your state’s Web site for both participation and performance data on your statewide assessments. Attached tables reflect what we believe to be the tests your state administers and the results that we have found thus far on the Web (Table 1), how participation information is reported for students with disabilities (if it is available) (Table 2), and whether information is given about students who took assessments with individual accommodations (Table 3).

Please review the table and verify its accuracy. Our goal is to (a) identify all components of each state’s testing system, (b) determine whether each state reports disaggregated test results for students with disabilities, (c) describe the way participation information is presented, and (d) describe how states report results for students who took the test with accommodations or modifications.

If any data element is inaccurate, please provide us with the public document and/or website that contains the accurate information. Address your responses to Hilda Ives Wiley at the above address.

If you have any questions about our request, please call Hilda Ives Wiley at (612) 626-8913 or email: ives0016@umn.edu. If we do not hear from you by Friday, June 27, 2003, we will assume that our summaries are accurate.

Thank you for taking the time to verify our findings. 

Sincerely,

Hilda Ives Wiley
Graduate Research Assistant 

Martha Thurlow
Director

Table 1: Tests Administered and Results Found

 Table 2: Participation Information for Students with Disabilities

Blank cell = No data

 Table 3: Accommodations

Please place a Yes or No into each box to describe information that is publicly reported.

Appendix B

Letters to State Directors of Special Education

(Two Forms Depending on Input from Assessment Director. Example here is if letter was verified by the Assessment Director. If no verification, letter was the same as in Appendix A.)

The National Center on Educational Outcomes is examining states’ public reports on 2001-2002 school year assessment results. We have reviewed your state’s Web site for both participation and performance data on your statewide assessments. Attached tables reflect what we believe to be the tests your state administers and the results that we have found thus far on the Web (Table 1), how participation information is reported for students with disabilities (if it is available) (Table 2), and whether information is given about students who took assessments with individual accommodations (Table 3). These results were verified by your state’s Director of Assessment, but if you have anything to add, please let us know.

Please review the tables and verify their accuracy. Our goal is to (a) identify all components of each state’s testing system, (b) determine whether each state reports disaggregated test results for students with disabilities, (c) describe the way participation information is presented, and (d) describe how states report results for students who took the test with accommodations or modifications.

If any data element is inaccurate, please provide us with the public document and/or Web site that contains the accurate information. Address your responses to Hilda Ives Wiley at the above address.

If you have any questions about our request, please call Hilda Ives Wiley at (612) 626-8913 or email: ives0016@umn.edu. If we do not hear from you by Friday, August 29, 2003, we will assume that our summaries are accurate.

Thank you for taking the time to verify our findings.

Sincerely,

Hilda Ives Wiley
Graduate Research Assistant

Martha Thurlow
Director

Table 1: Tests Administered and Results Found

Table 2: Participation Information for Students with Disabilities

Blank cell = No data

Table 3: Accommodations

Please change our No to a Yes if the information is publicly available.

Appendix C

2001-2002 State Assessment Systems and Status of Disaggregated Data

Appendix D

Disaggregated Participation Information

Appendix E

Participation Rate Analyses

Appendix F

Alternate Assessment Participation Information

Appendix G

Participation and Performance for Accommodations