Published by the National
Center on Educational Outcomes
Number 3 / August 2001
Directors of
Assessment, Special Education, and Title I! Mark Oct. 29!
Please
note the upcoming teleconferences on issues of inclusive standards, assessment, and
accountability. Among the topics addressed will be:
How
will we report results from our alternate assessment? And how will we integrate those
results into our accountability system?
Should
we go to online or computer based testing? What will that involve, and how will that
affect students with special needs, such as students with disabilities or with limited
English proficiency?
What
is “universal design of assessment” and can it help us overcome gray areas of
our assessment system?
Does
anyone know how to do this? What are other states doing?
These
are questions state directors of assessment, special education, and Title I are asking.
The answers to these questions require collaboration and joint planning across many
traditionally separate departments within state education agencies.
The
National Center on Educational Outcomes (NCEO), in collaboration with the network of
Federal and Regional Resource Centers, the Council of Chief State School Officers (CCSSO),
the National Association of State Directors of Special Education (NASDSE), and other
organizations to be announced, will be hosting quarterly teleconferences on topics related
to inclusive standards, assessment, and accountability to be followed by a time-limited
(2-4 weeks) online bulletin board on the same topic.
Our
first teleconference, on October 29, 2001, will be on the topic: Use of
Alternate Assessment Results in State Reporting and Accountability Systems.
Sue
Rigney from USDE, Title I, will be our primary presenter, with one state’s directors
of assessment, special education, and Title I serving as the reactants and discussants. We
will have a strict one hour limit to the calls, with about 20 minutes allotted to the main
presentation, 15 minutes to reactants, and up to 20 minutes for interaction. The
discussion can continue on an NCEO online bulletin board after the call. A summary of the
presentation, reaction, and discussion will also be available online after the call.
You
will be hearing more about this opportunity soon, from your state’s Regional Resource
Center, and from your state directors’ association. Circle the date on your
calendar—we will be getting the registration details out to you soon!
Graduation
Requirements and Diploma Options
The
last time NCEO surveyed states about their graduation requirements and diploma options was
in 1999. A lot has changed in two years, and it is important that we know what is going on
out there in these two areas. The National Center on Secondary Education and Transition
agreed to take leadership in gathering this information in 2001. The survey is being
conducted online (http://ici.umn.edu/hs/HSsurvey.html), but can also be completed
on paper.
Please
help us get this information up to date so that we are not giving out incorrect
information. If you need help completing the survey, please call Babette Gronland at (612)
626-0393, Dave Johnson at (612) 624-1062, or Martha Thurlow at (612) 624-4826.
Effects of
Read-Aloud Accommodation on Reading and Math Items
A
new study soon to be released by NCEO explores item difficulty estimates for reading and
math items administered with or without the read-aloud accommodation. In a unique approach
to analyzing extant data, this study defined four groups of students in a way that helped
to control for confounding effects that often occur when analyzing extant data.
The
four groups of students whose state multiple-choice test data were examined included: (1)
non-accommodated students with reading disabilities, (2) accommodated students with
reading disabilities, (3) non-accommodated students without disabilities who were selected
because their performance was similar to that of the students with reading disabilities,
and (4) non-accommodated students without disabilities who were randomly sampled from all
students without disabilities. These four groups provided controls for the typical
confounding of performance level and use of accommodation.
Item
difficulty for the first three groups was compared to that of group 4 to examine
differential item functioning (DIF). Because accommodations are considered to be a way to
improve the quality of measurement for students with disabilities, the number of DIF items
would be expected to be greatest for the first group (students with reading disabilities
who took the test without accommodations). In addition, few DIF items for the second group
would be expected (students with reading disabilities who took the test with
accommodations).
For
the Math test (32 items), there was one DIF item for the first group and six for the
second group. This finding suggests, as expected, that the reading accommodation makes
little difference in the item difficulty on a state mathematics assessment.
For
the Reading test (41 items), there were 10 DIF items for the first group and 19 DIF items
for the second group. This finding suggests that regardless of whether the students
received the accommodation, the difficulty for many items was different for students with
disabilities. This implies that better means of measuring reading skills for students who
struggle with reading are needed.
The
reading test results were further explored. The item difficulty differences were found to
be most pronounced for the last three items – always much more difficult for the
students with reading disabilities regardless of whether they had received the read-aloud
accommodation. When those items were removed from the analysis, the overall difference
between item difficulties was no longer significant for non-accommodated students with
reading disabilities. However, the difference continued to be significant for accommodated
students with disabilities, even when the last three items were removed.
There
clearly is need for further study of the effects of the read-aloud accommodation. Many
confounding factors may still affect the results, such as inconsistent administration of
the read-aloud accommodation, group administration of the read-aloud accommodation, and
inappropriate application of the accommodation (with some who needed it not receiving it,
and some who did not need it receiving it). These kinds of possibilities raise questions
about decision making and administration issues as well as simply the effects of the
read-aloud accommodation.
The
results of this study have several implications for current assessments:
Test
developers and item reviewers need to consider students with disabilities from the very
beginning—to ensure that assessments reflect the qualities of universal design.
Greater
attention must be devoted to making appropriate decisions about who needs certain
accommodations and who does not.
Read-aloud
and other accommodations that are subject to variability in administration must conform to
standards of administration that maintain consistency but allow for student management.
The
full report on this study is in Read-Aloud
Accommodation: Effects on Multiple-choice Reading and Math Items (Technical Report 31)
and will soon be on line at the NCEO Web site: (cehd.umn.edu/NCEO).