Special Education Teacher Responses to the 1997 Basic Standards Testing

Minnesota Report 14

Published by the National Center on Educational Outcomes

Prepared by Richard Spicuzza, Martha Thurlow, Ron Erickson, and Aaron Ruhland

September 1997

Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Spicuzza, R., Thurlow, M., Erickson, R., & Ruhland, A. (1997). Special education teacher responses to the 1997 Basic Standards testing (Minnesota Report No. 14). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/MnReport14.html

Overview

One of the primary challenges in implementing statewide testing is our ability to ensure that all children participate in the evaluation of student academic progress. Unfortunately, a current review of the literature indicated that between 40% and 50% of school-aged students with disabilities are not participating in educational evaluations (McGrew, Thurlow, Shriner, & Spiegel, 1992; Thurlow, Scott, & Ysseldyke, 1995). In fact, during the 1990 NAEP Trial State Assessment, 33% to 87% of students with disabilities were left out of these assessments (McGrew et al., 1992). Although most individuals within the field of education would consider these non-participation rates for students with disabilities to be too high, there is little, if any, information about: (a) how decisions are made, (b) who makes the decisions, and (c) what criteria are used to determine whether a child with disabilities should and is able to participate in typical statewide assessments.

Phillips (1995) suggests that despite the best intentions of policy makers and educators alike, “it will be necessary to balance the policy goal of maximum participation [of students with disabilities] against the need to provide valid and interpretable student test scores” (p. 6). Given this caveat, maintaining full inclusion during statewide testing provides at least two beneficial outcomes. First, educators and other constituents would be able to hold all children to the same standards (expectations); and second, it would increase our ability to ensure that all children are allowed to participate in equal educational opportunities (Phillips, 1995). The latter point actually is well founded in Federal law (EHA, 1975; IDEA, 1997) that guarantees students with disabilities a right to a “free, appropriate public education.”

On the other hand, all children may not be able to participate in the same manner given any one type of testing method (Thurlow, Olsen, Elliott, Ysseldyke, Erickson, & Ahearn, 1996). However, this does not mean that children should not or cannot be part of a statewide evaluation system. Thus, it is imperative for any state implementing statewide, high stakes assessment to evaluate the ramifications of how districts go about implementing and following through on state educational assessment mandates. Additionally, it will be important to evaluate how districts determine whether children with disabilities are included during regular testing cycles. While the issue of participation in statewide accountability systems is increasingly recognized, there is little research on how systems are actually operating. What we do know is that the decision about participation in assessments is generally made by the IEP team (Thurlow, Scott, & Ysseldyke, 1995). That is, 71% (32 of 42) of the states indicated that the IEP team made inclusion decisions for children with disabilities on statewide testing. Of this subset, 78% of the states report that IEP documentation must include a statement about the participation of the student in the statewide testing. To date this research has been based on the reports of state level personnel or state policies, not on what educators actually report. Similarly, we know some things about the use of accommodations — again from the reports of state level personnel and state policies. Typically, those in the field are not among the sources from which current research-based knowledge is obtained.

The purpose of this investigation was to explore how special education teachers make decisions to include children with disabilities. The 1997 administration of Minnesota's Basic Standards Test provided an opportunity for the Minnesota Assessment Project to examine how decisions were made to include or exclude students with disabilities. We targeted teachers to be the source of information, and were interested in: (a) who participated in the decision-making process, (b) what criteria were used to guide these decisions, and (c) when these decisions were made. Additionally, we asked the special education teachers to describe the type of accommodations students with disabilities used during the most recent testing cycle. Finally, teachers were asked to check additional testing accommodations — ones currently not allowed by state rule — from a survey checklist. This checklist was developed to inquire whether commonly used classroom accommodations would be potentially useful for evaluating special education students on the Minnesota Basic Standards Test.

Method

A survey was developed by University of Minnesota researchers in collaboration with the Department of Children, Families and Learning (CFL) (see Appendix A). Survey questions were presented in a variety of formats, including closed response, open ended, checklists, and a five-point Likert scale, depending on the specific issue addressed.

A total of 872 teachers from 28 school districts throughout the state of Minnesota were sent surveys during the months of April and May (1997) following the administration of the Basic Standards Tests. Districts were selected purposefully, to provide a cross section of different district sizes within the state of Minnesota. These 28 districts were to represent all regions of the state and to be representative of urban, suburban, and rural districts. Using a four district classification system provided by the CFL, districts were selected from: Cities of the first class (i.e., Duluth, Minneapolis, St. Paul), Suburban Metro (surrounding suburbs), Greater Minnesota districts with more than 2,000 students, and Greater Minnesota districts with less than 2,000 students.

After identifying the target districts, all special education teachers who provided instruction to eighth and ninth grade students in each of the 28 districts were sent a copy of the survey. Surveys were initially mailed to the district-appointed Graduations Standards Technician to disseminate further within the district. Surveys were returned to the University of Minnesota for analysis. Table 1 shows the number of schools recruited by district for the survey.

Table 1. Number of Schools Recruited by District

Findings

A total of 259 teachers (30%) returned the surveys to the University of Minnesota for further analysis. Of the 28 districts originally targeted, responses were received from 27 (96%) districts. Table 2 shows the number of teachers by district who responded to the survey.

Table 2. Return Rates of Teacher Surveys by District

Note: Percentages are based on the 259 returned surveys.

Participation

The first area addressed by the survey was: Who determined whether students with disabilities would participate in the Basic Standards Testing? An overwhelming majority of respondents indicated that IEP teams were making participation decisions. Additionally, 67% (n=173) of the special education teachers reported that the process for making participation decisions within their district was adequate compared to 31% (n=81) who did not. Although 45% (n=118) of the responding teachers noted that participation decisions were made primarily before January 1, 1997 and 13% (n=35) made the decision during a regular IEP meeting, 22% (n=55) of the remaining teachers indicated that they made decisions right up to the test date.

The second area addressed by the survey examined the influence of specific factors on the decision about whether to include students with disabilities in the Basic Standards Testing. Teacher responses were tabulated by analyzing responses on the four Likert-scaled questions. The Likert scale used “1” to indicate little to no influence and “5” to denote a high level of influence. It is evident by reviewing Table 3 that the severity of a student's disability was the most influential factor in the decision about whether a student would participate. Least influential was an external pressure to exclude the student.

Table 3. Rate of Influence on Determining Whether to Include Children with Disabilities in the Basic Standards Testing

Accommodations

The third area addressed by the survey was the type of accommodations students with disabilities were provided during the January, 1997 testing cycle of the Minnesota Basic Standards Test. Data are presented by overall frequency and percent of the 259 responding teachers (see Table 4). As is evident in this table, there was little differentiation in the accommodations provided for reading and for math. Timing/scheduling and setting accommodations were used most often, particularly extended time, small group administration, separate room administration, and allowing frequent breaks. The most frequent presentation accommodation was repeating directions and the most frequent response accommodation was answering in the test booklet.

The final section of the survey asked the special education teachers to identify instructional strategies or supports currently used within their classrooms that they believed may also prove beneficial to students with disabilities on the Basic Standards Test. Overall frequencies and percentages of responding teachers indicating a preference for each accommodation are presented in Table 5. As is evident in this table, there were three adaptations of instructional materials identified by more than half of the responding teachers: reducing the number of items (72%), highlighting key points (72%), and giving the student models of correctly completed work (68%).

Several testing supports that are not currently allowed were identified as potentially useful: interpretation of directions (69%), highlighting key words or phrases (67%), clarifying directions beyond script (63%), extending sessions over several days (63%), more frequent breaks (57%), providing additional examples (52%), and increasing spacing between items (51%). Most of these are presentation accommodations.

Table 4. Frequency (Percent) of Accommodations Used by Students with Disabilities Reported by Special Education Teachers

Note. Frequencies and percentages are based on the 259 returned surveys.

Table 5. Common Classroom Instructional Strategies Teachers Would Like To Include as Possible Accommodations on the Basic Standards Test for Students with Disabilities

Additional Comments

Throughout the survey a number of open-ended questions elicited additional comments from the special education teachers. Generally, teacher comments fell into one of three categories: Accommodations allowed and preferred, the overall Decision Making Process, and Recommendations. In addition, there were an array of additional comments that were classified as Miscellaneous. A brief description of these comments follows.

Accommodations. Teacher concerns in this area primarily addressed issues about the current audiotape versions of the Basic Standards Test. Specifically, teachers noted that the tapes appeared to jump around the test booklet, making it difficult for children to follow. Additionally, a few indicated that the pace of the tape did not match the student's pace in responding. An interesting comment that occurred on numerous surveys was that the accommodation checklist included on the survey was helpful. In fact, a few respondents suggested it was the first time they had been made aware of the multitude of accommodations available for special education students.

Decision-Making Process. Comments indicated that decision-making procedures often were not formal. Decisions were made “off the cuff,” and used blanket rules for inclusion (e.g., all LD students) or exclusion (e.g., all severe or moderately mentally impaired).

Recommendations. Two of the most frequent comments concerning recommendations for the future included: (a) providing a completely computerized version of the test for students with disabilities, and (b) establishing formalized rules to standardize inclusion rates across the state of Minnesota.

Miscellaneous. One of the more common statements across districts was that while the Basic Standards Test is not a timed test per se, this message does not get communicated to staff and students. Implicitly, students are forced to finish because a majority of their peers have completed the test and begin to move around.

Discussion

It is imperative for any state implementing high stakes assessment to evaluate the ramifications of how districts go about determining whether children with disabilities are included during regular testing cycles. To date, much of the information provided within the literature has been based on reports of state level personnel or state policies (Thurlow, Ysseldyke, & Silverstein, 1993; Thurlow, Scott, & Ysseldyke, 1995) and not on what educators report. Therefore, the purpose of this investigation was to examine how special education teachers made decisions to include children with disabilities in the most recent Minnesota Basic Standards Test (January, 1997). Specifically, we were interested in determining: (a) who participated in the decision-making process, (b) what criteria were used to guide these decisions, and (c) when these decisions were made.

Overall, the initial findings of this survey are encouraging. Current Minnesota state law mandates that the IEP team holds the ultimate decision-making authority for making participation decisions for students with disabilities. In fact, 67% (n=173) of the responding teachers indicated that current practice (relying on the IEP team to make inclusion decisions) is in line with state law. Additionally, teachers overwhelmingly denied that external pressure (e.g., public reporting) influenced their decision of whether to include or exclude a child with disabilities from the Basic Standards Testing. Furthermore, decisions appeared to be made in a timely manner, with 45% of the teachers reporting that these decisions were made before (prior to January 1) the January testing date.

On the other hand, there appeared to be lack of clarity on how to make inclusion decisions and a lack of a formal decision-making procedure that would help to standardize or provide a framework to educators who are faced with an array of critical decision points. Not only is there a global decision for IEP teams to contemplate, such as whether to include a child with disabilities, but there are additional considerations like the level of participation (e.g., state level, individual, or exemption), the type and kind of accommodation to provide, and whether any necessary modifications need to be in place. Over time these decision points will obviously become more and more a part of what IEP teams in the state of Minnesota will face. Additional investigations will need to examine how to best inform the public on the implications of these decisions and how to facilitate and create workable guidelines for the types of decisions IEP teams will be making.

Obviously the quality of a district's decision-making process will affect the overall participation rates of students with disabilities. The more formal and systematized the process, the greater confidence one can have in the uniformity of decisions within any given district. It would appear that further exploration is needed to identify whether a uniform decision-making process can be created (e.g., a decision-making rubric) and disseminated statewide to assist IEP teams.

There appears to be a healthy concern that additional accommodations may be necessary to encourage further participation, and the adequate assessment of students with disabilities. Future research and policy reviews should examine the potential areas for additional accommodations that do not adversely affect the reliability, validity, or test security of the assessment as a way to potentially increase the overall participation rate of students with disabilities in the Basic Standards Testing.

A review of the preliminary results from this survey suggest that overall, positive steps are being taken at the district level to facilitate inclusion of children with disabilities. While these first steps appear to be in the right direction, concerns remain. Though this investigation moved beyond state level reports or state policy guidelines to answer the three critical questions, it relied on teacher reporting of what occurred during the 1997 testing cycle. Thus, we rely on the accuracy of those recollections. In the future, researchers may wish to examine hard data (i.e., the state database) to address the following questions: (a) What are the participation rates for students with disabilities in the Basic Standards Tests of Reading and Math?, (b) What are the passing rates of students with disabilities on the Basic Standards Tests of Reading and Math?, (c) Is there a relationship between disability category and participation or passing rates?, (d) What are the subsequent success rates for students with disabilities who initially failed one or both of the Basic Standards Tests?, (e) What was the raw score distribution for students with disabilities on the 1997 Basic Standards Test?, and (f) Is there a range of first-time scores that predicts success on subsequent administrations of the Basic Standards Testing in Reading or Math?

Once answers are obtained for each of these questions, one would hope that more data-based policy decisions and participation decisions could be made. It is our intent that by gathering information for each of these questions and identifying the critical decision points IEP teams may face in the future, a more standardized approach can guide the process of determining participation and level of participation for students with disabilities. Overall, we believe answers to these questions are vitally important to policymakers and IEP teams charged with making participation decisions, but most importantly, to the children for whom a high school diploma depends on the impact of these findings and the policies that are instituted.

References

McGrew, K. S., Thurlow, M. L., Shriner, J. G., & Spiegel, A. N. (1992). Inclusion of students with disabilities in national and state data collection programs (Technical Report 2). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Phillips, S. E. (1995). All students, same test, same standards: What the new Title I legislation will mean for the educational assessment of special education students. Oak Brook: IL. North Central Regional Educational Laboratory.

Thurlow, M. L., Olsen, K., Elliott, J., Ysseldyke, J., Erickson, R., & Ahearn, E. (1996). Alternate assessments for students with disabilities (Policy Directions No. 5). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., Scott, D. L., & Ysseldyke, J. E. (1995). A compilation of states' guidelines for including students with disabilities in assessments (Synthesis Report 17). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., Ysseldyke, J. E., & Silverstein, B. (1993). Testing accommodations for students with disabilities: A review of the literature (Synthesis Report 4). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Appendix A

Minnesota Assessment Project 1997 Survey

Special Education Student Participation in Basic Standards Tests

1. Your district's name: ____________________________________________________

2. Primary disability category(ies) of students you teach __________________________

3a. Out of the total number of students you teach, how many are in the designated grade(s) that recently took the Basic Standards Tests in Reading and Mathematics? _______________

3b. Out of these eligible students, how many actually took one or both of the tests? ______

I. Participation Decisions

4. Do you think the process for deciding whether students with disabilities should participate in the Basic Standards Testing is adequate within your district?

Yes ____ No ____ If No, what changes do you think need to be made for the decision process to be more effective?_______________________________________________________

5. Who was involved in making participation/exclusion decisions? Please circle the most appropriate response:

(a) IEP Team                                                                                                         (f) regular education teacher only

(b) administration only (e.g., principal, superintendent, special ed. director)              (g) special educator only

(c) case manager only (h) no formal decision making process was used

(d) child only (i) Other? Please explain ________________________________________________________

(e) parent only

6. Please rate the influence that the following factors had on participation rates for students with disabilities in your district:

7. For the most recent testing cycle, when (i.e., time of year) were most participation decisions made for your eligible special education students? ________________________

II. Accommodation Decisions

8. How were, and who participated in, decisions about the type of accommodations to provide to your eligible special education students during the Basic Standards Testing?

________________________________________________________________________

________________________________________________________________________

9. Do you think the process for determining the type of accommodations a student should receive was adequate?

Yes ____ No ____

If No, what changes would you make for the decision process to be adequate? ________________________________________________________________________

10.  This table presents the current accommodations allowed by the state during Basic Standards Tests.  Please estimate the number of students with disabilities you teach who received each of the following accommodations.   Please record a zero if an accommodation was not used by any of your students.

11. From the list below, please check any strategies or instructional supports that you as a teacher use during typical instruction that may also be helpful for students with disabilities on the Basic Standards Tests. This list is not intended to be exhaustive; therefore, if there are additional options that you would like to see that are not listed please add them below.

Other Comments: _________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

Top of page