Prepared by:
Martha Thurlow
August 2001
Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:
Thurlow, M. (2001). Use of accommodations in state assessments: What databases tell us about differential levels of use and how to document the use of accommodations (TechnicalReport 30). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/Technical30.htm
This report is based on a paper presented at the annual meeting of the National Council of Measurement in Education, Seattle, Washington, April 11, 2001.
Since the enactment of IDEA 97, students with disabilities are being included in state and district assessments more than ever before, often with accommodations designed to give them access to the test. In the past, most states did not collect data on the use of accommodations. This is changing. As part of its most recent survey of states, the National Center on Educational Outcomes found that data were available from 12 states on the number of students using accommodations during state assessments. The extremely variable rates in accommodations use (8-82%) indicated that in some states more accommodations are being used than in others; still there was some relationship of accommodations use to level of schooling (elementary, middle, high school). Publicly reported data on accommodations use are unique to each state, making other comparisons difficult. Further study of how data on the use of accommodations are collected in states points to concerns about the accuracy of information and possibly the need for better methods for recording accommodations use.
For some time now, the participation of students with
disabilities and students with limited English proficiency in national, state,
and district assessments has been a topic of considerable discussion and
research effort, to the point that the National Research Council formed
committees and conducted studies to address the issues (e.g., August & Hakuta,
1999; McDonnell, McLaughlin, & Morison, 1997). These and numerous other efforts
were aligned with federal initiatives that require the participation of students
with disabilities (both IDEA 97 and Title I of IASA 94) and students with
limited English proficiency (Title I of IASA 94) in state and district
assessments.
Accommodations frequently are cited as one of the key
avenues for increasing the participation of students with disabilities in
national and state assessments (Anderson, Jenkins, & Miller, 1996; Elliott, &
Thurlow, 2000; Mazzeo, Carlson, Voekl, & Lutkus, 2000; Stancavage, McLaughlin,
Vergun, Godlewski, & Allen, 1996; Thurlow, Elliott, & Ysseldyke, 2000).
Generally defined, accommodations are changes in testing materials or procedures
that allow students to show their knowledge and skills rather than the effects
of disability or limited English proficiency. While there are a variety of other
terms in use (e.g., adaptation, modification), some of which may mean the same
thing and others of which may define the comparability or acceptability of a
given change (Elliott, Thurlow, Ysseldyke, & Erickson, 1997), the essence of the
concept is that either the materials or the procedures of testing have changed.
In this paper, I use “accommodations” as a generic term to cover all types of
testing changes.
Although the importance of accommodations for both
students with disabilities and English language learners is now recognized, the
requirement that accommodations be provided has a much longer history for
students with disabilities, a history entwined with the law (Section 504 of the
1993 Rehabilitation Act, Americans with Disabilities Act, IDEA). For the past
decade, the National Center on Educational Outcomes (NCEO) has examined the
issues surrounding the participation of students with disabilities in
large-scale assessments.
All states now have written policies guiding the
provision of assessment accommodations for students with disabilities (Thurlow,
House, Boys, Scott, & Ysseldyke, 2000). Indeed, assessment directors are much
more cognizant of their policies on assessment accommodations (Olson, Bond, &
Andrews, 1999), and many states are conducting or participating in research on
the effects of accommodations. Although our knowledge about what accommodations
are allowed in states and which are most controversial has increased
dramatically (see Thurlow & Bolt, in press-a), as has our recognition of the
research that has been conducted on a variety of test changes (Thurlow & Bolt,
in press-b; Tindal & Fuchs, 2000), this information tells us nothing about the
extent to which accommodations are used, nor about the specific accommodations
that are used most frequently.
The purpose of the present paper is to examine the use
of accommodations in state assessments by students with disabilities, and to
explore several related factors. Five primary topics are addressed:
1. To what extent do states have data on the use of assessment
accommodations by students with disabilities during state assessments?
2. How variable is the use of assessment accommodations during
state assessments by students with disabilities across states?
3. What information
exists on the use of specific accommodations?
4. What data are
reported publicly on accommodations used during state assessments?
5. How do states
collect data on the use of assessment accommodations by students with
disabilities?
Data collected in the past from states’ written policies
on assessment accommodations (Thurlow, Ysseldyke, & Silverstein, 1993; Thurlow,
Scott, & Ysseldyke, 1995; Thurlow, Seyfarth, Scott, & Ysseldyke, 1997; Thurlow,
House, Boys, Scott, & Ysseldyke, 2000), as well as information collected from
surveying state assessment directors (Olson, Bond, & Andrews, 1999; Roeber,
Bond, & Braskamp, 1997; Roeber, Bond, & Connealy, 1998) consistently indicate
that the most frequently allowed accommodations across states are (1) large
print, (2) Braille, and (3) reading directions. However, a look at the
distribution of disability categories among the school-age population (U.S.
Department of Education, 2000) suggests that these probably are not the most
frequently used accommodations. Most students have learning disabilities, speech
and language impairments, emotional or behavioral disabilities, and mental
retardation, not
visual disabilities, as the most frequently allowed accommodations might
suggest.
To better understand the use of assessment
accommodations, it is important to look at the extent to which states have data
on the use of accommodations by students with disabilities during state
assessments. Further, it is important to look at the extent to which students
with disabilities are using accommodations during state assessments.
Data on the use of accommodations were collected through
a survey conducted by NCEO during 1999 (Thompson & Thurlow, 1999). State
directors of special education, or their designees, in the 50 states were asked
to indicate whether their state assessment systems collected data on the use of
accommodations during assessments, and if so, to indicate the percentage of
students using accommodations for each state test.
Twelve states indicated that they collected data on the number of students with disabilities using accommodations in their state assessments. Further, they each supplied the actual percentage of students with disabilities using accommodations in their state assessments, for the grade levels in which they were administered. For summary purposes here, the percentages are averaged and grouped in Table 1 according to the level of school (elementary, middle, and high school).
Table 1. State-Reported Levels of Use of Accommodations
|
|
Elementary Grades |
Middle School |
High School Grades (9-12) |
Florida |
FL Writing Assessment |
51% (Gr 4) |
39% (Gr 8) |
34% (Gr 10) |
Indiana |
Statewide Assessment - Math |
28% (Gr 3) |
34% (Gr 6) |
80% (Gr10) 82% (Gr 10) |
Kansas |
KS Assessment Program – Math |
21% (Gr 4) |
14% (Gr 7) |
08% (Gr 10) |
Kentucky |
Kentucky Core Content Test |
82% (Gr 4) |
72% (Gr 7) |
50% (Gr 10) |
Massachusetts |
Comprehensive Assessment System |
61% (Gr 4) |
38% (Gr 8) |
25% (Gr 10) |
Maryland |
MSPAP - Reading |
53% (Gr 3) |
25% (Gr 8) |
|
Nevada |
Terra Nova Complete Battery |
51% (Gr 4) |
42% (Gr 8) |
44% (Gr 10) |
New York |
PEP Test – Reading |
50% (Gr 3) |
50% (Gr 6) |
|
Pennsylvania |
Reading and Math Assessment |
67% (Gr 5) |
52% (Gr 8) |
45% (Gr 11) |
Rhode Island |
Writing Performance Assessment |
49% (Gr 3) |
55% (Gr 7) |
60% (Gr 10) |
South Dakota |
Stanford Achievement Test
(Language, Math, Reading, Science, Social Science) |
63% (Gr2) 67% (Gr 4) |
59% (Gr 8) |
46% (Gr 11) |
West Virginia |
SAT 9 – Language, Math, Reading,
Science, Social Studies |
64% (Gr 3-11) |
|
|
As is evident in Table 1, all 12 states had data
available at the elementary level, while only 11 did at the middle school level,
and only 9 did at the high school level. The cells without data in Table I
correspond to the school levels for which accommodations data were not available
in Maryland, New York, and West Virginia.
In general, there is a downward trend in the percentage
of students using accommodations across grades. For the 16 tests with data on
percentages of students using accommodations for at least three grades, 11 show
a downward trend in percentages across school levels; 2 additional show a
downward trend from elementary to middle school, but minimal difference between
middle and high school. Thus, a downward trend either across two or three of the
three school levels is evident in over 95% of the possible comparisons. In the
other three cases where three levels of data were available on the same test,
the trends in percentages of students using accommodations show upward trends
from elementary to middle to high school; two of the three cases are from one
state (Indiana) while the other is from Rhode Island.
An examination of the relative levels of the percentages of students using accommodations also is of interest. Table 2 summarizes the state data in terms of the approximate percentage of students using accommodations at each school level. There are several states that hover consistently at either the lower, middle, or higher ranges of percentages. For example, Kansas is consistently within the lowest three percentage levels, regardless of school level. Only two states stay consistently above the 50% level, regardless of school level (Kentucky and Pennsylvania). Four states hover around the 40-60% range for at least two of the school levels (Nevada, Pennsylvania, Rhode Island, South Dakota).
Table 2.
Approximate Percentages of Students with Disabilities Using Accommodations at Each School LevelPercentage
|
School Level
|
||
Elementary
|
Middle
|
High
School
|
|
0 – 10% |
|
|
Kansas |
11 – 20% |
|
Kansas |
|
21 – 30% |
Kansas |
|
Massachusetts |
31 – 40% |
New York |
Florida |
Florida |
41 – 50% |
Florida |
Nevada |
Nevada |
51 – 60% |
Nevada |
Pennsylvania |
Kentucky |
61 – 70% |
Massachusetts |
|
|
71 – 80% |
|
Kentucky |
|
81 – 90% |
Kentucky |
|
Indiana |
91 – 100% |
|
|
|
Note: West Virginia is not included in this
table because its data were combined across all grades (3-11).
In addition, Maryland and New York are not included in the high school column
because they did not have data at that school level.
With the apparent increase in use of accommodations
during state and district assessments, there is interest not only in how many
students are using accommodations, but also in exactly which accommodations are
being used. This information will help us understand not only which specific
accommodations are used most, but also the extent to which combinations of
accommodations are used.
Three states that have collected data on specific
accommodations that are used by students with disabilities have either analyzed
or supplied data on accommodations to NCEO to analyze – Kentucky (Trimble,
1998), Rhode Island (Elliott, Bielinski, Thurlow, DeVito, & Hedlund, 1999), and
Missouri (Bielinski, Ysseldyke, Bolt, Friedebach, & Friedebach, in press). The
findings of these studies are presented here, and the results examined in
relationship to each other and to contextual characteristics of the states. Each
of these states has slightly different accommodations policies.
Kentucky
is a state with relatively broad accommodations policies. Districts are
instructed that assessment accommodations should be aligned with instructional
accommodations – those accommodations used during instruction may be used during
assessment. The exception is that reading the test aloud to students is not
appropriate when the test is a measure of reading decoding skills.
Rhode Island is one of a handful of states that allows
accommodations for all students. Students or their teachers must be able to
demonstrate need for a specific accommodation before it can be incorporated into
the testing procedures for the student. As a result of this uniquely inclusive
approach to accommodations, Rhode Island has important data about the numbers of
students with and without disabilities using accommodations.
Missouri
initiated its new Communication Arts assessment in 1998. Missouri coded the
specific accommodations used by students on bubble sheets at the time of
testing; also coded for these students was the student’s category of disability
and the disability area (math, reading, behavior). As a result of the richness
of its information, Missouri has important data about the numbers of students
with specific disabilities who used accommodations during state testing.
The data on specific accommodations used during
statewide assessment are collected in slightly different ways in each state,
which in turn affects the kind of information available on use of accommodations
during testing. Therefore, each state is presented separately here, with some
indication of how data on accommodations used were collected.
Kentucky. Although Kentucky’s accommodation policies have been in
place since 1992, it did not begin coding accommodations used during assessments
until 1994-95. For coding purposes, Kentucky groups its accommodations into
seven categories:
•
Reader/Oral. The assessment is read to the student in a way comparable to
the manner in which normal instruction is delivered.
•
Scribe/Dictation. The responses to the assessment are dictated by the
student and written down by a scribe in a way comparable to the manner in which
normal instruction is delivered.
•
Cueing. The student with disabilities uses mnemonics, problem-solving
organizers, semantic organizers, webs, or templates in responding to the
assessment in a way consistent with daily instruction and the student’s IEP or
504 Plan.
•
Paraphrasing. The content of the assessment is paraphrased for the student
with disabilities in a way comparable to the manner in which normal instruction
is delivered.
•
Interpreter. The content of the assessment is signed for the student with
disabilities in a way comparable to the manner in which normal instruction is
delivered.
•
Technological. Technology typically used by the student with disabilities in
daily instruction is made available during the administration of the assessment.
•
Other. Other accommodations normally made available in the delivery of
instruction are made available in the administration of the assessment.
(Trimble, 1998, pp. 23-24)
According to the 1999 NCEO survey data (see Table 1),
Kentucky is one of the states with the highest percentage of students using
accommodations during state testing (50–82%). The analysis of specific
accommodations used during two earlier testing years (Trimble, 1998) indicated
that across all students using accommodations, the most frequently used
accommodations were (not necessarily in order): Reader/Oral, Scribe/Dictation,
Paraphrasing, Oral & Scribe/Dictation, Paraphrasing & Reader/Oral &
Scribe/Dictation, Paraphrasing & Reader/Oral & Other, Paraphrasing & Reader/Oral
& Technology & Scribe/Dictation. Clearly, many students used combinations of
accommodations rather than a single accommodation. The percentages of students
using each of the most frequently used accommodations and combinations of
accommodations during 1995-96 are presented in Table 3.
The data in Table 3 support NCEO’s finding that the percentage of students using accommodations decreases across grades (Thompson & Thurlow, 1999). In addition, the data in Table 4 suggest that combinations of accommodations are used by larger numbers of students than are single accommodations: in grade 4, the largest percentage of students used the Paraphrasing & Reader/Oral & Scribe/Dictation, while in both grade 8 and grades 11/12, the largest percentages of students used Paraphrasing & Reader/Oral.
Table 3.
Estimated Percentages of Students in Kentucky Using Specific Accommodations
Accommodation |
Grade
|
||
4
|
8
|
11/12
|
|
Reader/Oral
|
|||
Scribe/Dictation |
2.51 |
---- |
---- |
Paraphrasing |
2.51 |
8.72 |
12.98 |
Reader/Oral & Scribe/Dictation |
18.65 |
3.92 |
---- |
Paraphrasing & Reader/Oral |
10.63 |
25.06 |
26.88 |
Paraphrasing & Reader/Oral &
Scribe/Dictation |
33.30 |
10.12 |
---- |
Paraphrasing & Reader/Oral &
Other |
---- |
---- |
---- |
Paraphrasing & Reader/Oral &
Technology & Scribe/Dictation |
2.16 |
---- |
---- |
Note: The percentages in this table are
estimated from the 1995-96 numbers presented by Trimble (1998) in Table 7
(accommodations use) and in Table 3 (number of students participating in
assessment). Cells with less than 100 students were not estimated. The numbers
in this table are considered to be estimates because it is possible that there
were discrepancies in coding the number of students actually using specific
accommodations or combinations, as well as some variation in the actual number
of students participating in the assessment.
Rhode Island. The policy of allowing any student for whom the need
for an accommodation could be demonstrated to use the accommodation during
testing began during the 1995-96 school year, at the same time that the state
first administered new performance assessments in Grade 4 mathematics, writing,
and health education. At that time, Rhode Island had a list of several dozen
accommodations that could be used by students.
According to the 1999 NCEO survey data (see Table 1),
Rhode Island is a state with moderate percentages of students using
accommodations during state testing (39-61%). The analysis of Rhode Island’s
data revealed that while students without
disabilities most often used just one accommodation, students with disabilities most often used between
one and four accommodations (Elliott, Bielinski, Thurlow, DeVito, & Hedlund,
1999).
Table 4 presents the percentages of students with and without disabilities using each of the most frequently used accommodations and combinations of accommodations. These data show that relatively small percentages of students without disabilities used accommodations; the largest percentage of these students used the timing accommodation. For students with disabilities, the percentage of students using timing only was comparable to the percentage of students without disabilities using timing. However, there were other accommodations used by percentages of students with disabilities that were just as high (Oral & Repeated Directions), and a couple of combinations of accommodations that had much higher percentages of students with disabilities using them (Oral & Repeated Directions & Setting; Oral & Repeated Directions & Setting & Response).
Table 4.
Estimated Percentages of Students in Rhode Island Using Specific Accommodations
Accommodation |
Students
with
|
Students without |
Oral
Presentation
|
1.62 |
0.81 |
Timing |
3.01 |
3.89 |
Oral &
Repeated Directions |
3.17 |
0.92 |
Oral &
Repeated Directions & Setting |
9.03 |
0.22 |
Oral &
Repeated Directions & Setting & Response |
7.80 |
0.25 |
Note: The percentages in this table are
estimated from 1995-96 numbers presented in Table 6 by Elliott et al. (1999).
They are considered estimates because there may have been discrepancies in
coding the number of students actually using specific accommodations or
combinations, as well as some variation in the actual number of students
participating in the assessment.
Missouri. In Missouri a list of about two dozen accommodations was
in place during the 1998 testing. IEP teams decided which accommodations
students would use, and then marked these on a Student Information Form, which
also contained information on the student’s disability category and area of
disability.
Missouri did not provide data to NCEO’s 1999 survey on the percentage of students using accommodations during state testing. However, Bielinski, Ysseldyke, Bolt, Friedebach, and Friedebach provided data on the percentages of students with disabilities who had valid test scores using accommodations in grade 3 (53%), grade 7 (53%), and grade 11 (33%). These data indicate that greater percentages of students use accommodations in elementary and middle school than in high school. Table 5 shows the percentages of students with various disabilities using the four most frequently used accommodations and combinations of combinations. These data were not broken down by grade by Bielinski et al. (in press).
Table 5.
Estimated Percentages of Students in Missouri Using Specific Accommodations
Accommodation |
Disability
|
||||
Learning Disability
|
Speech/ Language Disability
|
|
Emotional Behavior Disability
|
Sensory/
|
|
Small Group
|
10 |
10 |
|||
Small Group & Read Aloud
|
16 |
13 |
11 |
10 |
9 |
Small Group & Read Aloud &
Extended Time Each Session |
11 |
11 |
11 |
6 |
8 |
Small Group & Read Aloud &
Extended Time Multiple Choice and Dictation |
29 |
38 |
30 |
16 |
23 |
Note:
The percentages in this table are directly from Table 1 in Bielinski, Ysseldyke,
Bolt, Friedebach, and Friedebach (in press), and are based on only those
students who received valid scores on the Missouri Comunications Arts test.
When the Individuals with Disabilities Education Act
(IDEA) was reauthorized in 1997, it included a requirement that states report on
the number of students with disabilities included in state and district testing,
and the performance of students on those tests in the same way and with the same
frequency as they did for students without disabilities. NCEO started tracking
public education reports in 1997 (Thurlow, Langenfeld, Nelson, Shin, & Coleman,
1998; Thurlow, Nelson, Teelucksingh, & Ysseldyke, 2000; Ysseldyke, Thurlow,
Langenfeld, Nelson, Teelucksingh, & Seyfarth, 1998). These reports revealed that
few states reported any information about students with disabilities other than
enrollment data. Those that did report on the performance of students with
disabilities often did not make clear whether they had included only those who
took the assessments in the standard way, or had also included students who used
some kind of accommodation.
Of the 13 states that provided some type of test data in
1998, Ysseldyke et al. (1998) found that only Delaware included any information
on the use of accommodations. Delaware reported that “of the 3,395 special
education students who took the test, 1,033 had some type of accommodation and
2,362 tested with no accommodation” (Delaware State Department of Education,
1997, p. 25). This indicates that about 30% of students with disabilities in
Delaware used accommodations.
Oregon provided combined data on the numbers and
percentages of students who took the 1993-94 reading assessment with
modifications or who were exempt (Oregon Department of Education, 1997, p. 52).
It is not possible to separate modifications from exemptions. Further, the use
of the term “modifications” suggests that the report is focusing on the number
of students whose results were not being reported. How many students used
accommodations that were considered acceptable and thus were included in
aggregate scores cannot be determined.
When another review of state reports was conducted in
1999 (Thurlow, Nelson, Teelucksingh, & Ysseldyke, 2000), 13 states (but not
necessarily the same ones as in 1998) provided test-based data on students with
disabilities; 12 provided participation data. Only South Dakota provided
information about accommodations use (South Dakota Department of Education and
Cultural Affairs, 1999). South Dakota disaggregated the scores for students who
took the test under standard conditions, standard accommodations, and with
nonstandard accommodations. In a report that was released after NCEO’s data
collection period, North Carolina provided data on the numbers of students using
modifications for its Tests of Computer Skills for the classes of 2001 and 2002
(State Board of Education, 1999). In North Carolina, “modifications” is the term
used for both acceptable and not acceptable changes in testing conditions:
Students with disabilities or
students who are Limited English Proficient may take tests under modified
conditions provided the modifications are documented and do not violate the
validity of the tests. Modifications are used routinely during classroom
instruction. (State Board of Education, 1999, p. xi)
For both groups of students in the North Carolina
report, the most frequently used “modifications” were extended time
(approximately 42%), separate room testing (approximately 42%), and
administrator reads test aloud (approximately 26%). It is not possible to
determine from the North Carolina report how many students overall received
accommodations because data are reported by total number of accommodations used
and the number of each specific accommodation used.
An analysis is again underway at NCEO. This time we are
scanning all states’ Web sites as well as actual reports that they print. Data
on the use of accommodations during state testing were found for six states
(Colorado, Indiana, Louisiana, Kentucky, Nevada, and South Dakota).
Colorado provided numerous data summaries on its Web site (http://www.cde.state.co.us/). It provides the number of students using each accommodation, but does not distinguish between whether the students have IEPs, 504 plans, or neither (Colorado allows accommodations for all students based on documented need). Without a disability-based numerator and denominator, it is possible only to look at the relative use of specific accommodations in relation to the total number of students. Nevertheless, it is interesting to look at Colorado’s data because of its comprehensiveness (see Table 6). Two accommodations stand out as the most frequently used—timing changes (extended or modified) and oral presentation.
Table 6. Percentage of All Colorado Students Receiving
Accommodations*
|
Gr 3 Reading |
Gr 4 Reading |
Gr 4 Writing |
Gr 7 Reading |
Gr 7 Writing |
Gr 8 Math* |
Gr 8 Science* |
.06 |
.04 |
.01 |
.08 |
.03 |
.10 |
.05 |
|
Large Print |
.05 |
.06 |
.03 |
.03 |
.01 |
.08 |
.06 |
Oral Presentation |
2.14 |
2.51 |
2.67 |
2.88 |
2.92 |
3.10 |
3.18 |
Scribe |
.72 |
.77 |
.84 |
.31 |
.46 |
.24 |
.24 |
Signing |
.06 |
.05 |
.06 |
.06 |
.06 |
.06 |
.06 |
Asst Commun Dev |
.03 |
.02 |
.03 |
.01 |
.03 |
.01 |
.01 |
Timing Changes |
6.28 |
10.21 |
7.80 |
4.00 |
3.89 |
3.25 |
2.12 |
Note:
These numbers are estimates based on the total number of students tested and the
number of students using each accommodation.
* An additional accommodation, “use of number line” was used
by .03% of all students for math and by .03% of all students for science.
Indiana provided data on its Web site (http://doe.state.in.us/istep/) on the percentage of special education students and the percentage of 504 students who received accommodations during its statewide testing in 2000. These data are summarized in terms of percentages in Table 7. The increase in percentage of students using accommodations noted in earlier data (Thompson & Thurlow, 1999) again is evident in Indiana’s data.
Table 7. Percentages of Indiana Special Education Students
Using Accommodations
Assessment
|
Grade 3 |
Grade 6 |
Grade 8 |
Grade 10 |
English Language Arts |
56.24 |
77.75 |
77.84 |
84.42 |
Mathematics |
54.45 |
76.38 |
77.24 |
85.41 |
Kentucky also provided general data on the use of accommodations by students
with disabilities, but did not make a distinction between different content
areas; Kentucky’s Spring 2000 data, obtained from its Web page
(http://www.kde.state.ky.us/oaa/implement/) are for its norm-referenced test
only (CTBS/5). The percentage of students with disabilities using accommodations
at the end of primary, and in grades 6 and 9 were, respectively, 67.5%, 71.0%,
and 56.0%. Kentucky’s data again reflect the smaller percentage of students
using accommodations at the high school level that was noted by Thompson &
Thurlow (1999) and by Trimble (1998).
Louisiana also provided data to NCEO on students using accommodations during its norm-referenced testing in 2000. In its report, it distinguishes between students in special education and students on Section 504 plans for accommodations. In each case, it lists the number tested who are included in the averages (these are students who either tested without any accommodations or with the large print edition, transferred answers, or individual/small group administration), and the number tested who are excluded from the averages (these are students who tested with the accommodations of Braille, answers recorded, extended time, communication assistance, repeated directions, or tests read aloud—except for Reading Comprehension—or any other accommodations to align with the student’s daily instructional program). Table 8 shows the percentages of students included in the averages and excluded from the averages in Louisiana’s norm-referenced test data. The slight trend toward higher percentages included in higher grades corresponds to lower numbers of students tested in the higher grades.
Table 8. Students Included and Excluded from Louisiana
Averages Based on Accommodations Used
Student Group |
Grade 3
|
Grade 5 |
Grade 6 |
Grade 7 |
Grade 9 |
Special Education Students |
|
||||
Included in Averages |
5.7% |
5.8% |
6.1% |
7.7% |
11.2% |
Excluded from Averages |
94.3% |
94.2% |
93.9% |
92.3% |
88.8% |
Section 504 Students |
|
||||
Included in Averages |
12.8% |
15.4% |
18.2% |
20.1% |
22.6% |
Excluded from Averages |
87.2% |
84.6% |
81.8% |
79.9% |
77.4% |
Note:
Percentages are based on numbers tested in each grade. For special education
students, the numbers tested were 4924, 5185, 5592, 5506, and 4738 for grades 3,
5, 6, 7, and 9, respectively. For 504 students, the numbers were 3646, 3487.
3168, 2658, and 1513, respectively for the same grades.
Nevada
makes the distinction between regular testing conditions (which include students
taking the test with “permissible” accommodations) and special conditions (which
includes those using “special accommodations” that “impact the validity of the
comparison against the national norm sample.” The percentage of students with
IEPs using “not permissible” accommodations during Terra Nova testing was 22.7%,
27.3%, and 15.5% in grades 4, 8, and 10 respectively. [Nevada also reports on
numbers of students with disabilities not tested – 31.0%, 20.7%, and 15.1%.] It
is not possible to determine how many students with disabilities used approved
accommodations in Nevada’s Terra Nova testing.
South Dakota reported on the performance of IEP students on the SAT 9 during 1999 and 2000 (South Dakota Department of Education and Cultural Affairs, 2000). Data are separated into tables for standard administration, accommodated administration, and non-standard accommodations administration. South Dakota’s data provide an interesting comparison among percentages of students in each condition across grades and across years (see Table 9).
Table 9. Percentages of South Dakota Students Tested in Three
Conditions Across Two Years
Testing Condition |
Grade 2 |
Grade 4 |
Grade 8 |
Grade 11 |
||||
1999 |
2000 |
1999 |
2000 |
1999 |
2000 |
1999 |
2000 |
|
Standard |
38.1 |
46.1 |
32.6 |
34.7 |
40.1 |
36.7 |
55.4 |
50.8 |
Accommodation |
12.2 |
15.1 |
15.7 |
12.8 |
17.7 |
12.1 |
5.4 |
10.6 |
Non-Standard |
49.7 |
38.8 |
51.7 |
52.5 |
58.4 |
51.1 |
39.2 |
38.6 |
Note:
The percentages in this table were calculated from the total number of IEP
students tested – no data were provided on the number of students who were not
tested.
Although there are no requirements to keep track of
whether or which accommodations students use, there is interest in the field in
knowing how many students are using accommodations. Erickson, Ysseldyke, Thurlow,
and Elliott (1997) suggested that there are several benefits to be obtained from
documenting the use of accommodations, one of which is to provide information
for further research about the use of accommodations during testing. The recent
settlement of a suit brought by students with dyslexia and their families
supported the view that information on the use of accommodations should be
tracked (Advocates for Special Kids—“ASK” vs Oregon Department of Education) if
for no other reason than that by tracking the numbers we are also able to
monitor them to see that they are “appropriate.”
The fact that some states are already reporting data on
the use of accommodations (or modifications or special testing conditions, or
nonstandard administrations) indicates that these states have identified ways to
collect data on accommodations used during assessments. However, the data that
we have so far raise questions about how the data are being collected. There is
very little comparability in the data that are reported, suggesting that there
may be variability in the way that data are collected.
NCEO is conducting a survey of states that asks how
information is collected about the accommodations that students use on state
assessments. In responding to the NCEO’s online survey, states had the option of
picking one of five responses, or selecting “other” and writing out an
explanation. At the time that this paper was written, responses had been
obtained from all but five states.
The number and percentage of the 45 states selecting each of the options is shown in Table 10. As is evident here, nearly 50% of the responding states indicated that they had a form that is completed at the time of testing. Those states marking “other” gave explanations that fell into four basic types: (1) data collected at the IEP team meeting, then transferred into a state database to be connected later to test data; (2) some type of data collected on the test form, but not necessarily the accommodations themselves (three of these states indicated that either “standard” or “non-standard” was marked on the form); (3) data were collected in the past and will be collected again in the future, but were not collected this year; and (4) data maintained at the local level only. Four states noted that some indication of accommodations use was marked on the testing form, but not necessarily the specific accommodation.
Table 10. State Approaches to Collected Data on
Accommodations Use During Assessments
Response Option |
Number |
Percent |
No information collected. |
10 |
22% |
Form completed at time of testing indicates some or all
accommodations a student uses |
20 |
44% |
Form completed at time of testing indicates only one
accommodation student uses |
2 |
4% |
IEP information is coded on test form and verified at
testing |
5 |
11% |
Our state has not made a decision about this yet |
1 |
2% |
Other |
7 |
15% |
Note:
Percentages in this table are based on the 45 states responding to the NCEO
survey (Thompson & Thurlow, 2001) at the time this paper was presented.
The search for data on the use of accommodations in
state assessments has been quite a search! Where before there were no data,
there now are some data. The data that do exist at this point are of extreme
variability in kind and in quality. With the increasing attention being given to
the participation of students with disabilities in state assessments, and the
belief that part and parcel of this participation is the use of accommodations,
it is critical that we know more about the extent to which accommodations are
used, what kinds of accommodations are being used, and what is happening to
scores when accommodations are used.
It is clear from the data presented here that, as we
suspected, the accommodations that are most frequently allowed in statewide
testing are not the ones that are most frequently used. What we perhaps did not
already know is that there are some trends in the use of accommodations that may
need exploration. Why is there a downward trend in most states in the use of
accommodations as the grade level increases? Is it because students “grow out”
of their need for accommodations? Is it because they become embarrassed to use
accommodations that are available to them? Could it be that the environment
becomes less accommodating – that teachers at the upper grades are less willing
to provide accommodations in instruction and therefore they are no longer deemed
appropriate for assessment? Or, are the students who received accommodations in
the elementary grades no longer in school, and therefore no longer in need of
accommodations? Why do these same trends not exist in all states?
It is important to acknowledge that this paper is a
first attempt to look at the use of accommodations in more than a single state.
Attempting to do this opens up numerous issues, and certainly makes the findings
here preliminary at best. We know that some of the differences in use may be
related to states’ policies. Not only do some states allow more accommodations
than others, but some states consider certain changes to be accommodations (that
are counted) that other states (e.g., Kansas) consider to be part of natural
testing conditions and therefore not counted as accommodations used.
Still, the dramatic differences in the use of
accommodations raise additional issues for assessment programs. What is the best
way to collect data on the use of accommodations? Is this something that should
be marked on the testing form at the time of testing? Is the person
administering the test the best one to document accommodations, or is it the
student? Does this vary by age? Do we know that if an accommodation is marked it
is really used?
The question of how IEP team decisions about needed
accommodations are translated to assessment practice also emerges when thinking
about assessment accommodations for students with disabilities. The IEP team
often makes decisions about assessment accommodations several months before the
assessment is administered. How is the IEP team decision carried forward to the
day of testing? Is the decision always reflected in what happens during testing?
To the extent that research on the effects of accommodations uses extant
databases rather than experimental designs (Thurlow, McGrew, Tindal, Thompson,
Ysseldyke, & Elliott, 2000), these questions become significant issues in our
understanding of the effects of accommodations.
Finally, this look at available information on the use
of accommodations has highlighted another important issue. What is happening to
scores of those students who take assessments with accommodations? Few states
were as clear as Louisiana, so it is important to applaud Louisiana for
reporting its data in a clear and concise manner. Louisiana clearly
distinguished between those students who used accommodations on the state’s
norm-referenced test that allowed their scores to be aggregated with the scores
of other students and those who used accommodations that did not allow their
scores to be included. The “excluded” numbers were very large—ranging from 89%
to 94% of students in special education. A quick glance at the scores of the
students whose scores were not included indicated that most were below those of
the scores that were included. Is this happening elsewhere, where we cannot see
the data to know?
The continuing confusion surrounding the use of accommodations in assessments needs resolution. The first step toward that is to be sure that the data that are produced by states reveal what is really happening. Considering how best to code accommodations that students actually use during testing is one step in that direction.
Anderson, N. E., Jenkins, F. F., & Miller, K. E. (1996).
NAEP inclusion criteria and testing accommodations: Findings from the NAEP 1995
field test in mathematics. Princeton, NJ: Educational Testing Service.
August, D., & Hakuta, K. (Eds.). (1997). Improving schooling for language-minority
children: A research agenda. Washington, DC: National Academy Press.
Bielinski, J., Ysseldyke, J., Bolt, S., Friedebach, M.,
& Friedebach, J. (in press).
Prevalence of accommodations for students with disabilities participating in a
statewide testing program. Diagnostique.
Delaware Department of Education. (1997). State summary report: 1997 Delaware writing
assessment program. Dover, DE: Author.
Elliott, J., Bielinski, J., Thurlow, M., DeVito, P., &
Hedlund, E. (1999). Accommodations and the
performance of all students on Rhode Island’s performance assessment (Rhode
Island Report 1). Minneapolis, MN: University of Minnesota, National Center on
Educational Outcomes.
Elliott, J. L., & Thurlow, M. L. (2000). Improving test performance of students with
disabilities in district and state assessments. Thousand Oaks, CA: Corwin
Press.
Elliott, J. L., Thurlow, M. L., Ysseldyke, J. E., &
Erickson, R. N. (1997). Providing
assessment accommodations for students with disabilities in state and district
assessments
(Policy Directions 7). Minneapolis, MN: University of Minnesota, National Center
on Educational Outcomes.
Erickson, R., Ysseldyke, J., Thurlow, M., & Elliott, J.
(1997). Reporting educational results for
students with disabilities (NCEO Policy Directions 8). Minneapolis, MN:
University of Minnesota, National Center on Educational Outcomes.
Mazzeo, J., Carlson, J. E., Voekl, K. E., & Lutkus, A.
D. (2000). Increasing the participation of
special needs students in NAEP: A report on 1996 NAEP research activities
(NCES 2000-473). Washington, DC: U.S. Department of Education, Office of
Educational Research and Improvement.
McDonnell, L., McLaughlin, M., & Morison, P. (Eds.).
(1997). Educating one & all: Students with
disabilities and standards-based reform. Washington, DC: National Academy
Press.
Olson, J., Bond, L., & Andrews, C. (1999).
Data from the annual survey: State student assessment programs. Washington,
DC: Council of Chief State School Officers.
Oregon Department of Education. (1997). 1996 status report: Special education,
student services, and compensatory education. Salem, OR: Author.
Roeber, E., Bond, L., & Braskamp, D. (1997).
Annual survey of state student assessment programs: Fall 1996 (Data on 1995-96
statewide student assessment programs). Washington, DC: Council of Chief
State School Officers.
Roeber, E., Bond, L., & Connealy, S. (1998).
Annual survey of state student assessment programs: Fall 1997 (Data on 1996-97
statewide student assessment programs). Washington, DC: Council of Chief
State School Officers.
South Dakota Department of Education and Cultural
Affairs. (1999). South Dakota annual
report of academic progress. Pierre, SC: Author.
Stancavage, F., McLaughlin, D., Vergun, R., Godlewski,
C., & Allen, J. (1996). Study of exclusion and assessability of students with
disabilities in the 1994 Trial State Assessment of the National Assessment of
Educational Progress. In National Academy of Education Panel on the Evaluation
of the NAEP Trial State Assessment, Quality and utility: The
1994 Trial State Assessment in Reading.
State Board of Education. (1999). Report of student performance: North Carolina Tests of Computer Skills.
Raleigh, NC: Division of Accountability Services/Testing Section.
Thompson, S .J., & Thurlow, M. L. (1999).
1999 state special education outcomes: A
report on state activities at the end of the century. Minneapolis, MN:
University of Minnesota, National Center on Educational Outcomes.
Thompson, S. J., Thurlow, M. L., & Boys, C. (2001).
2001 state special education outcomes: A report on state activities at the
beginning of a decade. Minneapolis, MN: University of Minnesota, National
Center on Educational Outcomes.
Thurlow, M. L., & Bolt, S. (in press-a). Web-based summary of research on testing
accommodations. Minneapolis, MN: University of Minnesota, National Center on
Educational Outcomes.
Thurlow, M. L, & Bolt, S. (in press-b). Accommodations most frequently allowed in
states—What we know from policy and research (Synthesis Report).
Minneapolis, MN: University of Minnesota, National Center on Educational
Outcomes.
Thurlow, M. L., Elliott, J. L., & Ysseldyke, J. E.
(1998). Testing students with
disabilities: Practical strategies for complying with district and state
requirements. Thousand Oaks, CA: Corwin Press.
Thurlow, M. L., House, A., Boys, C., Scott, D., &
Ysseldyke, J. (2000). State participation
and accommodations policies for students with disabilities: 1999 update
(Synthesis Report 33). Minneapolis, MN: University of Minnesota, National Center
on Educational Outcomes.
Thurlow, M. L., Langenfeld, K. L., Nelson, J. R., Shin,
H., & Coleman, J. E. (1998). State
accountability reports: What are states saying about students with disabilities?
(Technical Report 20). Minneapolis, MN: University of Minnesota, National Center
on Educational Outcomes.
Thurlow, M. L., McGrew, K. S., Tindal, G., Thompson, S.
J., Ysseldyke, J. E., & Elliott, J. L. (2000). Assessment accommodations research: Considerations for design and
analysis (Technical Report 26). Minneapolis: University of Minnesota,
National Center on Educational Outcomes.
Thurlow, M. L., Nelson, J. R., Teelucksingh, E., &
Ysseldyke, J. E. (2000). Where’s Waldo? A
third search for students with disabilities in state accountability reports
(Technical Report 25). Minneapolis, MN: University of Minnesota, National Center
on Educational Outcomes.
Thurlow, M. L., Scott, D. L., & Ysseldyke, J. E. (1995).
A compilation of states’ guidelines for accommodations in assessments for
students with disabilities
(Synthesis Report 18). Minneapolis, MN: University of Minnesota, National Center
on Educational Outcomes.
Thurlow, M. L., Seyfarth, A., Scott, D. L., & Ysseldyke,
J .E. (1997). State assessment policies on
participation and accommodations for students with disabilities: 1997 update
(Synthesis Report 29). Minneapolis, MN: University of Minnesota, National Center
on Educational Outcomes.
Tindal, G., & Fuchs, L. (1999). A summary of research on test changes: An
empirical basis for defining accommodations. Lexington, KY: University of
Kentucky, Mid-South Regional Resource Center.
Trimble, S. (1998). Performance trends and use of accommodations
on a statewide assessment: Students with disabilities in the KIRIS on-demand
assessments form 1992-93 (MD/KY Report 3). Minneapolis, MN: University of
Minnesota, National Center on Educational Outcomes.
U.S. Department of Education. (2000). To assure the free appropriate public
education of all children with disabilities: Twenty-second annual report to
Congress on the implementation of the Individuals with Disabilities Education
Act. Washington, DC: Author.
Ysseldyke, J. E., Thurlow, M. L., Langenfeld, K.,
Nelson, J. R., Teelucksingh, E., & Seyfarth, A. (1998). Educational results for students with
disabilities: What do the data tell us? (Technical Report 23). Minneapolis,
MN: University of Minnesota, National Center on Educational Outcomes.