Published by the National
Center on Educational Outcomes
November, 2004
NCEO continues to study the public reporting of state assessment participation and performance data on students with disabilities. It has been doing this since 1998, when its first study found that only 11 states publicly reported disaggregated results for students with disabilities. NCEO’s reports have evolved over time, as assessment systems themselves have changed. For example, NCEO now checks for the reporting of participation and performance information for alternate assessments as well as for general assessments. The most recent addition to NCEO’s reports is information on whether states report data for accommodated assessments.
The NCEO series of reports on public reporting of disaggregated participation and performance data is as follows:
All of these reports are on the NCEO Web site. The newest report, Technical Report 39, covers the 2001-2002 school year. It reveals that 35 states reported on both the participation and performance of students with disabilities for their general assessment, up from 28 states for 2000-2001. For the alternate assessment, participation and performance reporting was available in 22 states, up from only 13 states in 2000-2001.
Recommendations. With the increased data in the 2001-2002 reports, NCEO was able to make several recommendations for reporting. Most of the recommendations are directed toward Web-based reporting since most states now are reporting primarily on their Web sties. The recommendations were as follows:
Next Report. A report containing information on 2002-2003 participation and performance reporting is now in the verification process. NCEO verifies by sending lists of the reports and data that it has found to both the state director of assessment and the state director of special education. A non-response indicates to NCEO that the list is accurate. The verified data are used to create the report indicating which data are available.
The Out-of-Level Testing Project, funded by the Office of Special Education Programs, is ending September 30, 2004. The project produced 15 reports that cover research on out-of-level testing policy, technical considerations, and the implementation of out-of-level testing. During the last year of funding, the project updated its analysis of state out-of-level testing policies and produced case study reports based on districts in which out-of-level testing was implemented.
States’ Out-of-Level Testing Policies for 2003-2004. The publication, Rapid Changes, Repeated Challenges: States’ Out-of-Level Testing Policies for 2003-2004 (Report 13) (VanGetson, Minnema, & Thurlow), details changes in out-of-level testing policies over the past three years. In 2000-2001, 12 states were using out-of-level tests to measure student progress toward content standard proficiency. In 2003-2004 the number of states using out-of-level tests had increased to 17 states. However, some states had eliminated out-of-level testing, while new states had added this approach to testing.
States attempting to comply with the requirements of No Child Left Behind (NCLB) have tried to include all subgroups of students in assessment. Several states utilized out-of-level tests as a means of including more students in statewide assessment. However, given the federal mandate that all students must receive challenging, grade-level curriculum to support acquisition of grade-level content standards, states now are forced to look critically at their large-scale assessment policies regarding out-of-level testing.
Out-of-Level Testing Case Study. The publication, Understanding Out-of-Level Testing in Local Schools: A First Case Study of Policy Implementation and Effects (Report 11) (Minnema, Thurlow, & Warren), highlights several themes that emerged from interviews with special education teachers. The study found that some out-of-level test results were not available in a usable test score format because the test contractor did not analyze out-of-level test scores that were more than one level below the grade of enrollment. Since there is no normative information, students’ performance can’t be compared to the performance of other students. Less than 1/3 of the test scores were included in the state’s accountability indexes, primarily because state policy did not allow test data from assessments more than one grade level below the students’ grade of enrollment.
The results also indicated inconsistencies of policy implementation within and between schools, and concern that students who are tested out-of-level appear to be experiencing lost opportunities to learn. Teachers of students tested out of level consistently indicated that these students were not instructed on the grade level in which they were enrolled.
The authors stress that out-of-level testing policies continue to undergo major revisions, and caution that results cannot be generalized to other schools or districts. However, the findings do point to key concerns about implications of out-of-level testing policies.
Several state partners and researchers are involved in planning the upcoming China-U.S. Conference on “Aligning Assessment with Instruction,” scheduled for July 11-14, 2005 in Beijing, People’s Republic of China. Conference topics include alignment framework, technical adequacy and fairness, developing aligned assessments, using results, and professional development to support alignment. Issues related to culturally and linguistically diverse populations may be included in each of the topics.
For more information about the conference, and to submit papers and presentations, visit http://www.globalinteractions.org. Deadline for submissions is November 15, 2004.
Plans are being made with our partner organizations to make sure that this year’s NCEO Teleconferences are the best ever—to meet states’ needs in this era of reform and increased accountability pressure. Look for the exact dates and topics in the near future.
Please visit NCEO’s online survey and answer questions about what you think of NCEO’s Web site, how useful you find its content and resources, and suggestions you might have to improve it. Go to: http://cehd.umn.edu/NCEO/websurvey.htm. Thanks for your help!