Feasibility and Practicality of a Decision Making Tool for Standards Testing of Students with Disabilities

Minnesota Report 21

Published by the National Center on Educational Outcomes

Prepared by Bonnie Swierzbin, Michael E. Anderson, Rick Spicuzz, Lynn Walz & Martha L. Thurlow

December 1999


This document has been archived by NCEO because some of the information it contains is out of date.


Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Swierzbin, B., Anderson, M. E., Spicuzza, R., Walz, L., & Thurlow, M. L. (1999). Feasibility and practicality of a decision-making tool for standards testing of students with disabilities (Minnesota Report No. 21). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/MnReport21.html


Overview

Large-scale assessments, and in particular high stakes graduation tests, are becoming common throughout the United States. The 1997 reauthorization of the federal Individuals with Disabilities Education Act (IDEA) sets the expectation that nearly all students with disabilities will participate in large-scale assessments for the purpose of measuring students’ progress toward state standards beginning in 1998. It also supports the development of appropriate test adaptations for students with disabilities. In addition, states must report the number of students with disabilities participating in the assessments and the performance achieved by the students.

In Minnesota, statewide accountability testing currently takes place in grades three, five, and eight. The tests given at grades three and five are called Minnesota Comprehensive Assessments (MCAs) and cover the areas of reading, mathematics, and writing. The MCAs are given solely as a measure of system accountability. The eighth grade tests are called Basic Standards Tests (BSTs) and have a dual role. Like the MCAs, they are used for systemic accountability, but they are also used as minimum standards that students must pass in order to be eligible for a high school diploma. The BSTs are made up of tests of reading and mathematics administered beginning in eighth grade and a writing test administered in 10th grade.

In order to continue to receive federal special education funds, states are required to include all students with disabilities in their regular assessments, with accommodations as needed. An important part of planning for appropriate participation and accommodations for students with disabilities is having a set of stable testing guidelines that educators can refer to so that decisions are made on a consistent basis across students, schools, and districts. By 1997, most states had developed assessments and guidelines for use in planning accommodations for students with disabilities. Minnesota has written guidelines for the use of accommodations for the Minnesota Basic Standards Tests (BSTs) and for the Minnesota Comprehensive Assessments (MCAs). Both sets of guidelines, which are distributed to schools, have been available primarily in paper format.

Despite the historical practice of pervasive exclusion of students with disabilities from assessment and accountability systems across the United State, there are states that have made significant efforts to include all students and to report the performance of students with disabilities in statewide tests (Thurlow, Langenfeld, Nelson, Shin & Coleman, 1998). School districts across Minnesota have been including nearly 90% of eighth graders with disabilities in the BSTs (Thompson, Thurlow, Spicuzza, & Parsons, 1999) and Minnesota’s participation rates for students receiving special education services in the statewide MCAs are among the highest in the United States (Thompson et al., 1999).

As in other states, Minnesota’s written guidelines have been distributed to schools across the state. Guidelines also can be found on the Department of Children, Families, and Learning Web site. By themselves, however, written guidelines in standard text format have not been sufficient to help students with disabilities, who along with their parents, teachers, and administrators, make informed decisions about whether they should participate in testing and what accommodations should be used.

Informal discussions with educators have revealed that they did not believe that they had received enough information to make informed test participation decisions for students with disabilities. Furthermore, many educators indicated that they found the paper format difficult to understand when applying guidelines to complex decisions. Educators expressed frustration in not knowing who should make participation decisions, and were unfamiliar with the test accommodations and modifications available to students with IEPs or 504 plans (Spicuzza, Erickson, Thurlow, & Hurley, 1996; Spicuzza, Thurlow, Erickson, & Ruhland, 1997; Ysseldyke, Thurlow, McGrew, & Shriner, 1994; Ysseldyke, Thurlow, McGrew, & Vanderwood, 1994). This type of frustration may be increased as guidelines are revised and updated for each test cycle; educators become uncertain about which version is current.

With this frustration in mind, Minnesota Assessment Project researchers designed a decision making tool to inform special educators of the most up-to-date participation and test accommodation guidelines and to assist school staff in the process of making informed decisions with students and their families (Spicuzza, Erickson, Thurlow, & Hurley, 1996). To make the tool interactive and widely available, it was designed to be used on-line or on a computer disk. This report describes the decision making tool and reports on the results of a study that examined the tool’s feasibility and practicality.

 

Creation of a Decision Making Tool

To create a tool that would be helpful for making decisions about participation in assessments and accommodations, it was decided that the presentation of the written guidelines needed to be simplified. This was accomplished by designing a presentation format that divided the assessment guidelines into smaller portions of text.

There are two reasons to simplify the presentation of the guidelines. First, the new presentation format enables someone who is not an expert in assessment to use the decision making process for planning. Second, sequenced guidelines can help parents, teachers, administrators, and student work together to make the best participation decision for the student.

Other issues related to using a paper format for the assessment guidelines surface with teachers when discussing accommodation decisions for statewide testing. During the implementation of this new accountability system, the Minnesota graduation standards have caused many memos and information reports to be generated and a specific guideline easily gets lost or misplaced before the administrator or teacher receiving the information can plan how to use the news. Therefore, the assessment guidelines are not available or ready to use when needed by the planning team. Also, it is difficult to keep documents current and assure that everyone on the decision making team has the same information.

In order to reduce these difficulties and make the decision making process more efficient, a decision making tool was designed that could be accessed on the World Wide Web. The guidelines and a decision matrix (see Appendix A) were integrated into the Web-based tool. This tool was created in a Web-based format because the complexities of test decision making can be presented in an understandable and manageable manner. The Web uses a combination of graphics and text that divide the decision matrix into a series of questions for consideration during a planning discussion.

A format was needed that would be easily accessible to all people who might be making testing decisions. This includes teachers, administrators, parents, students, and other IEP team members. While the availability of computer technology and access to the World Wide Web varies greatly from district to district, and even from building to building and home to home, most schools and families have some access to the Web or will in the near future. A paper format of guidelines will always be needed; however, the Web-based version of the guidelines has several advantages. With the Internet, decision making tools such as this can be available to everyone in the state and changes in the process can be made instantly since everyone is accessing the same copy. Using the Internet also allows teachers and parents to have access from home or other community sites.

It should also be noted that one does not have to be connected to the World Wide Web in order to use the decision making tool; access to a Web browser (such as Netscape or Microsoft Explorer) is all that is needed. If one has a Web browser, but is not connected to the Web, the decision making tool can be run from a diskette. This format does not allow the copy to be instantly updated, but it does give educators and parents access to the decision making matrix.

Finally, the Web tool was designed to be simple in terms of graphics and other features. This allows for quick access, making it easy to maneuver through the site. The design is also straightforward. Since educational teams are often making decisions with students, the decision matrix is short and simple. It was designed so that people with varying technological backgrounds can become familiar with the tool very quickly.

 

 The Decision Making Tool

The decision making tool used in this study is an electronic version of Minnesota guidelines for making test participation decisions and a decision matrix (see Appendix A). The address for the decision making tool is http://www.coled.umn.edu/ NCEO/MAP/ (see Appendix B) The Web site design has two branches, one for students with disabilities and another for students with limited English proficiency; both are centered on making decisions about inclusion and the use of accommodations in statewide testing. For each of these parallel branches users choose which type of test is being planned for: (1) Minnesota Comprehensive Assessment (statewide accountability testing) or (2) Basic Standards Tests (basic graduation requirement test). The homepage for the decision making tool ends with this statement and decision.

This web site was designed to help parents and educators make decisions about student participation in different forms of statewide testing. Follow the links below to a decision matrix for Limited English Proficient (LEP) students or students with IEP or 504 plans.

The Web site design has two branches of decision making to reflect the dual role of the tests within both the LEP and IEP/504 plan branches—one for statewide accountability testing for grades three, five, and eight, and one for graduation requirements testing for grades eight and beyond. The reason for having these two lines of decision making is that the decision may vary according to the reason a student is taking a test, either for statewide accountability or individual achievement for graduation. The second page for both the LEP and IEP/504 branches ends with the following statement and decision.

Follow the links below to see if a student should participate in either form of testing and what kinds of accommodations are available for each test.

Graduation Standards         Statewide Accountability Testing

As the user begins the decision making matrix, a question is posed about the student for whom the test participation decision is being made. The person using the decision matrix responds to this question by clicking on the appropriate answer, LEP or IEP/504. Then the second question asks about which type of test a decision is being made for. Next, the user is asked about student status to help determine whether the student should be included in the standardized test. If the student should be included, the user is given information about accommodations; if not, the user is given information about alternate forms of assessment. At any point in the process, the user can go back a step or to the beginning of the decision process. In the end, the user is provided with guidance on how a student could be included in the standardized test so that the student can best demonstrate knowledge within the state guidelines.

 

Decision Making Tool Flowchart

The flow of questions for the Web-based decision making tool; guidelines for accommodations in testing is as follows:

Decision Making Tool Flowchart


Method

To determine whether a computer-based decision making tool is useful for people involved in making decisions about the participation of students with disabilities in the Basic Standards Tests and Minnesota Comprehensive Assessments, a survey was developed for parents, educators, and administrators who participate in the decision making process. Survey response formats included closed response and open-ended response (see Appendix C).

In December 1998 and January 1999, a total of 102 surveys were sent to 6 districts across the state of Minnesota. These districts were chosen from a larger set of districts that had previously agreed to work with CFL during the development and administration of the Basic Standards Tests. Some districts that had been recommended were not able to participate due to of a lack of computer resources needed to use the decision making tool.

CFL classifies all districts in the state of Minnesota into one of four categories based on size and location. The categories are:

• Cities of the First Class (large, urban school districts)

• Suburban-Metro

• Greater Minnesota > 2000 (rural districts with more than 2,000 students)

• Greater Minnesota < 2000 (rural districts with fewer than 2,000 students)

Four of the districts included in this study were in greater Minnesota with a population over 2,000 students, and two districts were in suburban-metro areas.

A special education coordinator in each of the selected districts was sent the World Wide Web address of the decision making tool as well as a copy of the tool on a floppy disk, formatted for Macintosh or IBM PC-compatible computers, whichever was appropriate for the district. Thus, survey respondents had two ways to access the tool: through the Web site if they had Internet access, or from the disk if they had a Web browser but no Internet access.

For each district, a special education coordinator received 16 surveys to distribute, plus one extra survey for making copies if needed. The special education coordinator was asked to distribute the surveys to educators and parents who were making decisions about the participation of students with disabilities in the Basic Standards Tests or Minnesota Comprehensive Assessments. After surveys were completed, the coordinator was asked to collect the surveys and send them to NCEO for analysis.

Once surveys were returned, they were numbered and all quantitative responses were entered into a computer database for analysis. For the qualitative responses, one member of the research team used an inductive qualitative research technique to develop a coding system as described by Bogdan and Biklen (1992). All of the qualitative data were reviewed holistically for regularities and themes. A system of coding categories based on the themes was developed and verified; all of the participants’ responses were clustered under these categories.


Results

Fifteen surveys (14.7%) were returned from three of the six districts (50%). The individual return rate of 14.7% was very low; however, extra surveys had been sent to each coordinator, who also had the option of distributing fewer surveys than were sent or copying the surveys and distributing more. In addition, a number of respondents used the tool to make decisions for more than one student before completing the survey.

Respondents were primarily Special Education teachers and parents. One administrator and one school psychologist also completed the survey. Table 1 shows the breakdown of respondents by position.

 

Availability and Familiarity of Computer Technology

Of the six districts that were chosen to participate in the study, two (33%) were initially unable to do so because they lacked the computer technology needed to use the decision making tool. In both cases there was no computer available where the decision making process took place, or the computer that was available lacked the software necessary to run the decision making tool. One of these two districts was loaned a Macintosh laptop computer by the Minnesota Assessment Project in order to be able to use the tool and complete the survey.

Table 1.  Position Held by Survey Respondents

Position

Number of Respondents

Percent of Total

Special Education Teacher 8 53%
Parent 5 33%
Administrator 1 7%
School Psychologist 1 7%
Total 15 100%

Nearly one-half of the respondents (47%) used the tool on a Macintosh computer, another 27% used the tool on a PC, and 20% did not specify what type of computer they used. One respondent (6%) reviewed the tool after participating in a meeting in which someone else used the tool to lead the discussion.

Respondents were asked about their familiarity with computers in general, with the specific type of computer (Macintosh, PC) they used to look at the tool, and with the Internet. Table 2 shows the number of respondents who indicated their level of familiarity with computers in general, the specific type of computer used for this study, and Internet use. Almost half of the respondents (47%, 7 of 15 responses) indicated they were very familiar with computers in general. Another 33% (5 of 15) were somewhat familiar with computers, two respondents (13%) were a little familiar with computers and one respondent (7%) was not at all familiar with computers before using the decision making tool. None of the parents among the respondents reported being very familiar with computers in general. Of the 15 respondents who used the tool on a computer, 33% were very familiar with the type of computer they used the tool on, 27% were somewhat familiar, 33% were a little familiar, and 7% were not at all familiar with it. Of these same 15 respondents, 27% were very familiar with the Internet, 27% reported being somewhat familiar, 27% were a little familiar, and 20% were not at all familiar with the Internet.

In general, 87% (13 of 15) respondents who used the decision making tool on a computer reported having the computer skills needed to use the tool. One respondent reported not having the required computer skills "for the 1st student, but the next two were better."

Overall, 80% of the respondents (12 of 15) reported that they have access to a computer at home. Of these respondents, 58% (7 of 12) indicated that they have Internet access at home. All 10 of the respondents who work in the school districts indicated that they have computer access at school and have Internet access there. Just over half (8 of 15, 53%) of the respondents have access to a computer where their IEP team meets, another 27% (4 of 15) do not have access, and the remaining 20% did not answer the question.

Respondents were asked what Internet browser they have access to. The responses were almost evenly divided between Netscape Navigator (4 of 7, 57%) and Microsoft Explorer (3 of 7, 43%). Of the 10 respondents who had Internet access at their schools, six (60%) have only Netscape Navigator, three (30%) have only Microsoft Explorer, and one (10%) has both. Of the six respondents who have Internet access where the Individualized Education Plan (IEP) team meets, two (33%) have Netscape Navigator, three (50%) have Microsoft Explorer, and one did not reply to this question.

When asked whether their school keeps IEPs on computer, nine of the ten educators indicated that they did; 80% said all IEPs were on computer, 10% said some IEPs were on computer and one educator did not answer the question.

 

Participation Decisions

Respondents were asked who in their school district usually makes decisions about the participation of students with disabilities in Basic Standards Tests and Minnesota Comprehensive Assessments. Although there were six responses to choose from, only three responses were selected (see Table 3). Nine respondents (60%) indicated that the IEP team makes the decision (the special educators, one parent, and a psychologist); one respondent identified a special education teacher, a classroom teacher, and parents as decision makers. The second most common response after "IEP team" was "I don’t know" with 27% of responses (4 of 15) falling in this category. The four respondents who gave this answer were all parents. One respondent (7%) indicated that a general education administrator usually makes participation decisions.

 Table 2. Respondents' Familiarity with Computer Technology (N=15)

  General computer experience Experience with the computer used Internet Experience
Very Familiar 7 5 3
Somewhat Familiar 5 4 5
A Little Familiar 2 5 4
Not at all Familiar 1 1 3

 

Using the Decision Making Tool

Slightly more than one-half of the respondents (8 of 15, 53%) used the decision making tool during an IEP team meeting. Another 20% (3 of 15) used the tool before the IEP team meeting in which a decision was made, and 20% (3 of 15) used the tool after making a decision in an IEP team meeting. One respondent (7%) did not answer this question.

Overall, ten respondents (67%) looked at the computer-based tool as part of a decision making group. Another four respondents (27%) looked at it alone, and one respondent used it both as part of a group and alone.

Respondents in the school districts were asked how many students they used the tool to make participation decisions for. Responses to this question varied from 3 to 0 (one special educator reviewed the tool but did not use it to make a decision). Table 4 shows the breakdown of respondents by the number of students for whom they made decisions using the tool.

Respondents were asked approximately how long it took for them to go through the tool for the first student and the last student for whom they used the tool. For the first student, the respondents’ answers ranged from two to twenty minutes while for the last student, they ranged from one to ten minutes (See Table 5). The average time to use the tool for the first student was 10.5 minutes, and the average for the last student was 5.4 minutes. Two of the respondents reported not using the tool to make decisions about student participation. Six respondents reported using the tool to make decisions for only one student (4 parents, psychologist, and administrator). The seven respondents who reported using the tool for more than one student were special educators. In all but two situations, the time needed to use the tool decreased by half or more from the first to the last use.

Table 3.  Decisions about Participation in the MCA and BST

Decision Maker Number of Respondents Type of Respondents
IEP Team 10 8 Special Educators, 1 Parent, & 1 Psychologist
I don't know 4 Parents
Administrator 1 Administrator

 

Overall, 67% (10 of 15) of respondents thought the format of the computer-based tool was easy to understand. Another 27% (4 of 15) thought that it was not easy to understand (all parents); one of these respondents commented that they were "not familiar with Standards." One respondent (7%) thought the format of the Web version of the tool was easy to understand, but was unable to understand how to use the disk version. Only one of the five parents who answered the survey thought the format of the tool was easy to understand. Respondents were also asked if the tool was easy to use. A majority (10 of 15, 60%) thought it was easy to use. One respondent again had difficulty with the disk version of the tool but not the Web version. Again, only one parent thought it was easy to use.

 

Table 4.  Number of Students for Whom the Decision Making Tool was Used for Planning

Students Planned Number of Respondents % of Respondents
3 3 20%
2 4 27%
1 7 47%
0 1 7%

Respondents were asked whether they found the tool helpful in making decisions for a particular type of student. Eight of fifteen respondents (53%) thought it was helpful; they mentioned students with the following types of disabilities: learning disability, mild mental impairment, questionable ability, and students requiring modifications. One respondent (7%) did not think the tool was helpful, and the remaining six (40%) did not answer the question. Of the six who did not answer the questions, five were parents who said they could not answer for another parent about a student who was not their child.

When asked whether they would use the tool again to make participation decisions for students with disabilities, 53% (8 of 15) respondents said that they would do so. Another 13% (2 of 15) said they would not use it again. The special educators comments were, "For one student it wasn’t necessary and wasted time. It was good when I did it ahead of the meeting," "waste of time–easier to do with paper flow charts," and from a parent, "should be a teacher, not parent decision." An additional 27% (4 of 15) were not sure whether they would use it again. One said, "Some IEP teams are in too much of a hurry to wait for this," while a parent said that further usage of the tool would be someone else’s decision.

Eleven respondents (73%) said they would recommend the tool to others who are involved in participation decisions. Three respondents (20%) were unsure whether they would recommend it (all parents). Another 7% (1 of 15) would not recommend the tool because the process is "easier to do with paper flow charts." Respondents were asked when in the decision making process they thought the tool would be most helpful. Eight of fifteen (53%) thought it would be helpful as preparation for an IEP team meeting, two (13%) thought it would also be helpful during the meeting, and 5 of 15 (33%) did not know. Of the five who did not know whether the tool would be helpful for an IEP meeting, four were parents.

Table 5.  Time Needed to use the Tool

Minutes for First Student's Plan Minutes for Last Student's Plan No. of Respondents
No response No response 2
2 No response 1
3 1 1
5 2 1
5 3 1
5 10 1
10 7 1
10 No response 1
12 No response 1
15 5 1
15 No response 2
20 10 1
20 No response 1
10.5 avg 5.4 avg.  

 

Qualitative Results

In addition to the quantitative results discussed so far, there were a number of comments written in answer to open-ended questions and voluntarily written on the surveys. A list of the comments was compiled and separated into the following topic categories:

1. Clarity of information in the tool

2. Usefulness of the tool

3. Mode in which information is presented

4. Amount of information in the tool

5. Technology issues

6. Clarity of the survey

7. Miscellaneous

Major themes of each category are discussed below. (For a complete list of comments, see Appendix D.)

• Clarity of information in the tool.
The comments in this category fell into two major groups: Special education professionals said that the information was "simple and clear" and "very useful and thorough." In general, their reactions were positive. On the other hand, parents of special education students made comments that the information "should be written much more clearly and not in educational jargon," and it should be "easier to understand." It seems that none of the five parents had a positive reaction to using the tool.

• Usefulness of the tool.
Respondents thought that the tool was useful for some, but not all, types of students. One commented, "For some students a simple discussion is enough and this is not necessary," while another said it was useful for students of questionable ability. A respondent also pointed out that "some IEP teams are in too much of a hurry to wait for this." The usefulness of the tool is also related to the mode in which information is presented to decision-makers, the topic category that will be discussed next.

• Mode in which information is presented.
The major theme in this category is that many people, particularly parents of students with disabilities, would like to have information on paper. One comment sums up the situation very well: "We used the tool during our meetings. Perhaps if we had gone through the process before the meeting and printed out the results, or if we provided a companion written piece for parents, it would have worked better."

• Amount of information in the tool.
Some respondents wanted to see more information about modifications to Basic Standards Tests in the tool.

• Technology issues.
Respondents in one school district found the Web site easy to understand but were unable to use the disk version of the tool. Other respondents pointed out that in large IEP team meetings not everyone could see the computer screen.

• Clarity of the survey.
Some respondents were confused by the fact that the tool refers to Basic Standards Testing and Statewide Accountability Testing while the survey used both those terms and the acronyms BST and MCA.

• Miscellaneous.
Comments in this category included one person wondering whether the participation of students with disabilities in Basic Standards Tests and Minnesota Comprehensive Assessments would be required in the future. Another was a statement from a parent that such participation should be a teacher’s, not a parent’s decision.


Discussion

The purpose of this study was twofold: (1) to create a tool that presented the CFL guidelines for making decisions about participation of students with disabilities in the statewide testing, and (2) to determine the usefulness of this tool for decision makers (e.g., teachers, parents, administrators, students). In order to make the decision making process efficient and accessible, a decision making tool was designed using the World Wide Web.

Six school districts agreed to participate in a field test of the tool and provide feedback through a user survey. Four of the districts were greater Minnesota schools with student populations over 2000 and two of the districts were in suburban metro locations. The data from these districts support several general conclusions.

Although most respondents reported that a team made the participation decision, it is unclear from this survey who should be team members. It is noted here that an IEP team appropriately includes parents, classroom teachers, special education teacher, administrator, and student when appropriate (U.S. Department of Education, 1997). No surveys were submitted from classroom teachers. Since most students with disabilities are included in general classes it is important to systematically include the general education teachers in the development of the IEP (Gilliam & Coleman, 1981, Nevin, Semmel, & McCann, 1983). Although the audience for the decision making tool varies for each district, a special educator is always involved. Therefore, it is reasonable to conclude that the special educator is a primary audience for this tool.

Parents seem to be confused and frustrated by the process of testing and decision making about testing, and the tool added to that confusion rather than diminishing frustration. It is possible that parents’ confusion about the decision making tool was confounded by other factors. It appears that, overall, the tool was not helpful for parents involved in test participation decisions. Several studies have found that there is limited involvement of parents in the IEP process (Gartner & Lipsky, 1992; McLaughlin & Warren, 1995; Singer & Butler, 1992). Parents who are attending IEP meetings need tools that will promote an assurance of appropriate education for their child, but not tools that result in confusion. Some knowledge of statewide testing is assumed of the audience that would use the decision making tool as it now stands. This tool will need revision for parents to benefit from its use in decision making.

In summary, when testing accommodations are made available so that as many students as possible can meaningfully participate in an assessment, it is important to have efficient and accessible guidelines available for the people making accommodation decisions. These people may include administrators, teachers, counselors, parents and students. When testing decisions are being made on an individual basis, guidelines for making these decisions need to be clear so that decisions are made fairly and accommodations are used to help make the tests more accessible to the student. When students receive the accommodations they need, the validity of the test results increases.

At the present time, most states do not have a decision making process in place that will walk people through this often-complicated process. The decision making tool designed for educators in Minnesota and available through the Internet is an attempt at helping streamline this process. Although the tool contains language that some people find confusing, and the tool may not be accessible to some due to technological availability, it does offer many benefits. Most special education professionals found it to be a clear and useful resource. As technology increases in schools and homes around the state, this tool should become more accessible. The decision making tool is efficient, taking a brief amount of time, and thorough. Using the information gained through the survey data contained in this report, the decision making tool can be improved so that it is even more useful to educators.


References

Bogdan, R. & Biklen, S. (1992). Qualitative research for education. Boston: Allyn and Bacon.

Friedson, W. S. (1990). The individualized education program conference: An investigation of factors associated with parent involvement. Kent State University. Pro Quest [on-line service] (Doc. No. AAC9015874).

Gartner, A. & Lipsky, D. K. (1992). Beyond special education: Toward a quality system for all students. Harvard Educational Review, 57, 367-395.

Gilliam, J. E. & Coleman, M. C. (1981). Who influences IEP committee decisions? Exceptional Children, 47, 642-644.

Goldstein, S., Strickland, B., Turnbull, A. P., & Curry, L. (1980). An observational analysis of the IEP conference. Exceptional Children, 46, 278-286.

McLaughlin, M. J., & Warren, S. H. (1992). Individual education programs: Issues and options for change. College Park, MD: the Center for Policy Options in Special Education, University of Maryland at College Park.

Nevin, A., Semmel, M. I., & McCann, S. (1983). What administrators can do to facilitate the regular classroom teacher’s role in implementing individual educational plans: An empirical analysis. Planning and Changing, 14, 150-169.

Singler, J. & Butler, J. (1992). The Education for All Handicapped Children Act: Schools as agents for school reform. In T. Hehir and T. Latus (Eds.), Special education in the century’s end: Evolution of theory and practice since 1970. Harvard Educational Review 159-190.

Spicuzza, R., Erickson, R., Thurlow, M., & Hurley, C. (1996). Focus Group Input on Students with Disabilities and Minnesota’s Basic Standards Tests (State Assessment Series, Minnesota Report 3). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Spicuzza, R., Thurlow, M., Erickson, R., & Ruhland, A. (1997). Special Education Teacher Responses to the 1997 Basic Standards Testing (State Assessment Series, Minnesota Report 14). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., Langenfeld, K. L., Nelson, J. R., Shin H. & Coleman, J. E., (1998). State accountability reports: What are states saying about students with disabilities? (Technical Report 20). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thompson, S., Thurlow, M., Spicuzza, R., & Parson, L. (1999) Participation and performance of students receiving special education services on Minnesota’s Comprehensive Assessment: Read and Math and Writing; 1998 through 1999 (State Assessment Series. Minnesota Report 18). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thompson, S., Thurlow, M. L., Spicuzza R., & Parsons, L. (1999). Participation and performance of students receiving special education services on Minnesota’s Basic Standards Tests: Reading and Math, 1996 through 1998 (Minnesota Report 18). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

U.S. Department of Education (1997). Annual Report to Congress on the Implementation of P. L. 94-142. Washington, DC: Office of Special Education Programs.

Ysseldyke, J. E., Thurlow, M. L, McGrew, K. S., & Shriner, J. G. (1994). Recommendations for making decisions about participation of students with disabilities in statewide assessment programs: A report on a working conference to develop guidelines for statewide assessment and students with disabilities (Synthesis Report 15). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes

Ysseldyke, J. E., Thurlow, M. L., McGrew, K. S., & Vanderwood, M. (1994). Making decisions about the inclusion of students with disabilities in large-scale assessments: A report on a working conference to develop guidelines on inclusion and accommodations (Synthesis Report 13). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.


Appendix A

Overview of Decision Matrix

Overview of Decision Matrix


Appendix B

Minnesota Assessment Project Decision Making Tool: 

Graduation Standards and Statewide Accountability Testing

http://www.coled.umn.edu/nceo/MAP/


Appendix C

Computer-Based Decision Making Tool Survey

 

Terms: BST = Basic Standards Tests (grade 8+)

MCA = Minnesota Comprehensive Assessments (grades 3 & 5)

 

Part I: Tell us about yourself

 

1. Who are you? (Please check one.)

a. ___ parent d. ___ regular classroom teacher

b. ___ student e. ___ administrator

c. ___ special education teacher f. ___ other (please describe)

_______________________

_______________________

 

2. Is your school district: (Please check one.)

a. ___ urban (e.g., Minneapolis, St. Paul, Duluth)

b. ___ suburban-metro (e.g., Bloomington, Eagan, Roseville)

c. ___ a greater Minnesota district with more than 2,000 students

(e.g., Rochester, St. Cloud, Bemidji)

d. ___ a greater Minnesota district with fewer than 2,000 students

(e.g., Mountain Lake, Owatonna, Windom)

 

3. How did you look at the decision-making tool? (Check one.)

a. ___ on computer; what kind?

b. ___ on paper

c. ___ I never looked at it (Another person used the tool.)

 

4. How familiar were you with the following equipment before you used the decision-making tool?

a. A computer in general (circle one number choice)

1. not at all familiar

2. a little familiar

3. somewhat familiar

4. very familiar

 

b. The specific type of computer (IBM, Macintosh) you used to look at the tool (Circle one number choice.)

1. not at all familiar

2. a little familiar

3. somewhat familiar

4. very familiar

5. I didn’t look at it on a computer

 

c. The Internet (Circle one number choice.)

1. not at all familiar

2. a little familiar

3. somewhat familiar

4. very familiar

5. I didn’t look at it on a computer

 

5. Did you have the computer skills you needed to use the decision-making tool? (Check one.)

a. ___ Yes

b. ___ No; please explain ______________________________________

c. ___ I didn’t look at it on a computer (Someone else used the tool.)

 

6. Do you have access to a computer in the following places? (Check one answer for each letter.)

a. at home

1. ___ yes If yes, do you have Internet access? a. ___ yes

b. ___ no

2. ___ no

 

b. at school

1. ___ yes If yes, do you have Internet access? a. ___ yes

b. ___ no

2. ___ no

 

c. where your IEP team meets

1. ___ yes If yes, do you have Internet access? a. ___ yes

b. ___ no

2. ___ no

 

7. If you answered yes to any of the parts of question 6, which Internet browsers (e.g. Netscape Navigator, Microsoft Explorer) are available on this machine?

a. (home)

b. (school)

c. (IEP team)

d. ___ I don’t have access to a computer

 

8. Does your school keep Individualized Education Plans (IEPs) on computer? (Check one.)

a. ___ yes; how many? 1. ____ all IEPs

2. ____ some IEPs

b. ___ no

c. ___ I don’t know

 

Part 2: Tell Us About the Decision-Making Tool

9. In your school or district, who usually makes decisions about the participation of students with disabilities in the Basic Standards Tests or Minnesota Comprehensive Assessments? (Check one.)

a. ___ a special education administrator

b. ___ a general education administrator

c. ___ a special education teacher

d. ___ a parent/parents of the special education student

e. ___ the IEP team

f. ___ other; please explain ____________________________________

 

10. When did you look at the tool? (Check all that apply.)

a. ___ before the IEP meeting at which we used the tool to help make decisions

b. ___ during the IEP meeting at which we used the tool to help make decisions

c. ___ after the IEP meeting at which we used the tool to help make decisions

d. ___ I never looked at the tool. (Someone else used the tool.)

 

11. When you looked at the tool, were you: (Check one.)

a. ___ alone

b. ___ part of a group

c. ___ I never looked at the tool. (Someone else used the tool.)

 

12. For how many students did you use the tool to help make BST and MCA participation decisions? (Check one.)

a. ___ zero d. ___ three g. ___ more than five

b. ___ one e. ___ four

c. ___ two f. ___ five

 

13. How many minutes did it take for you to go through all the parts of the tool and make decisions for the first student for which you used it? (Fill in the blank with an approximate number of minutes, or fill in "NA" if you did not use the tool.)

__________ minutes to go through it for the first student

 

14. If you used the tool for more than 1 student, how many minutes did it take for you to go through all the parts of the tool for the last student for which you used it? (Fill in the blank with an approximate number of minutes, or fill in "NA" if you did not use the tool.)

__________ minutes to go through it for the last student

 

15. Was the format of the tool easy to understand? (Check one.)

a. ___ yes

b. ___ no; why not? ___________________________________________

c. ___ I never looked at the Decision-making tool. (Someone else used the tool.)

 

16. Was the tool easy for you to use?

a. ___ yes

b. ___ no; why not? ___________________________________________

c. ___ I never looked at the decision-making tool. (Someone else used the tool.)

 

17. What additional information about participation in BST and MCA testing would you like to see included in this tool? (Write your answer below.)

 

18. Was this tool more useful for making decisions about a particular type of student? (Check one.)

___ yes; what type of student? _________________________________

___ no

 

19. In the future, would you use this tool again with the same student or with another student? (Check one.)

___ yes

___ no; why not? ___________________________________________

___ I’m not sure; why?

 

20. Would you recommend this tool to others making these decisions?

___ yes

___ no; why not? ___________________________________________

___ I’m not sure; why?

 

21. In your opinion, would this tool be most helpful: (Check one.)

___ As preparation for an IEP team meeting

___ During an IEP team meeting

___ After an IEP team meeting

___ I don’t know

 

22. How can this tool be improved so that it is more useful to IEP teams making decisions about the participation of Special Education students in BSTs and MCAs? (Please write your thoughts below.)

 

23. What questions do you still have about the participation of Special Education students in the BST and MCAs? (Please write your thoughts below.)

 


Appendix D

Survey Comments from Respondents

 

Words in italics were added to make comments more clear.

A "P" following a statement indicates a parent offered the comment.

 

Clarity of information in tool

It is pretty good the way it is.

I thought it was very useful and thorough. I think it helped our team understand that there are steps involved in making decisions.

Good information

I thought it was simple and clear. It asks all the questions needed to make a clear objective decision especially when considering exempting.

This tool was very helpful to learn more about grad. standards—especially accommodations. The other teachers at our school are now using the web address. We enjoyed it. Even our grad. standards technician was interested and mentioned this site for all of the teachers.

More clearly explained. P

Should be written much more clearly and not in educational jargon. P

Easier to understand. P

 

Usefulness of tool

(Useful for) students with modifications.

(Useful for) LD with modifications.

(Useful for) LD.

(Useful for) MMI.

(Useful for) when the decision is in doubt.

For some students a simple discussion is enough and

this is not necessary.

(Useful for students with) questionable ability.

Waste of time–easier to do with paper flow charts.

For one student it wasn’t necessary and wasted time. It was good when I did it ahead of the meeting.

Some IEP teams are in too much of a hurry to wait for this.

I felt this gives a better understanding of how or what accommodation can/should be used.

Easier to do with paper flow charts.

 

Mode in which information is presented

Helpful to have booklet available for parents to provide explanation. P

Previous written information made available to parents. P

Parent needs more time to read information beforehand. P

Inservice for SPED

Several parents requested a handout. Most of our parents don’t understand grad. standards at all. The parents don’t seem real happy to do the surveys.

It would be more useful if it printed a form at the end listing the accommodations for the student and other information.

We used the tool during our meetings. Perhaps if we had gone through the process before the meeting and printed out the results, or if we provided a companion written piece for parents, it would have worked better.

 

Amount of information in the tool

Include some ideas for modifications on the BST in the software

 

Technology issues (time, visibility)

The IEP team seemed somewhat impatient about waiting for this process as part of the meeting. Our meetings are large and not everyone could see the screen.

The Website (format was easy to understand), not the disk—couldn’t understand how to use it.

Disk didn’t work.

 

Clarity of the survey

I don’t know what this question refers to, the acronyms especially. (refers to BST and MCA) P

What are BST & MCA’s? P

What are these – BST and MCA? P

 

Miscellaneous

We just don’t know if they will ever be required in the future. So it’s hard to get excited about it. (response to question 23 about the participation of students with disabilities in the BSTs and MCAs)

Should be a teacher, not parent decision. P