Prepared by Richard Spicuzza, Ronald Erickson, Martha Thurlow, and Aaron Ruhland
This document has been archived by NCEO because some of the information it contains is out of date.
Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:
Spicuzza, R., Erickson, R., Thurlow M. L., & Ruhland, A. (1996). Input from the field on assessing students with disabilities in Minnesota's Basic Standards Exams (Minnesota Report No. 1). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/MnReport1.html
Nationally, there is a strong push for higher standards of learning and the implementation of assessment programs to measure progress toward these higher standards (for example, see publications by the American Federation of Teachers, 1995; the Business Roundtable, 1996; the Education Commission of the States, 1996; and Education Daily, July 23). As states and schools implement these standards and assessments, they are faced with the challenge of ensuring that they include all students, particularly students with disabilities and students with limited English proficiency (LEP), who frequently are excluded from these types of educational initiatives (c.f., McGrew, Thurlow, Shriner, & Spiegel, 1992).
Currently, the State of Minnesota is facing the challenge to define high standards and to assess students' progress toward these articulated standards. In addition, policies and guidelines are being developed, at a state and local level, to encourage maximum participation of all children. Before 1994, there was no requirement in Minnesota for any kind of state level assessment. While individual school districts have generally collected their own data in the past, the nature of the assessment varied from one district to another. Additionally, there was no attempt to provide a comparative, statewide picture of student performance for the public.
In response to federal and State legislation, however, Minnesota is developing an assessment system that will provide state-level reports on the performance of its students. The assessment system includes both Basic Standards Examinations, designed to assess basic skills, and Profiles of Learning, designed to assess high-level instructional standards of learning. This two-tiered approach is an effort to ensure that Minnesota students meet both Basic Skills requirements and challenging standards before graduating from high school.
The Basic Standards Exams for Reading and Mathematics were first implemented throughout the state on a voluntary basis during the 1995-1996 school year. School districts were encouraged to participate in the assessment process and to include all eligible students (i.e., of appropriate age). The most recent testing cycle in April 1996 was both legislated and conducted within a short-time span. Thus, relatively sparse guidelines were provided to districts about how to include students with disabilities in the Exams or about possible accommodations students could receive to encourage maximum participation.
The recent implementation of the Basic Standards Exams within the state of Minnesota provided an opportunity for the Minnesota Assessment Project to examine: (a) overall participation rates of students with disabilities; (b) what kinds of accommodations were made available; (c) specific needs of districts to ensure they are able to include as many students as possible in these assessments; and (d) retrospectively, how decisions were made to include or exclude students with disabilities.
To answer these questions, the University of Minnesota, in conjunction with the Minnesota Department of Children, Families and Learning, developed and distributed a survey to twenty-two districts across the state. Surveys were mailed out to target districts and returned to the University for analysis. These data were perceived as being an important first step before gathering more in-depth information from district personnel through focus groups on the participation of students with disabilities in the Basic Standards Graduation Exams.
Surveys were distributed to 22 districts across the state of Minnesota. The districts were chosen to include a mix of urban, metropolitan, and rural settings. Each district had participated in the April 1996 administration of the Basic Standards Exams. The assessment coordinator in each of the 22 districts was contacted by phone to explain the purpose of the survey. Surveys were sent to districts by mail, and returned to the University of Minnesota for analysis.
The survey consisted of eight questions about the participation of students with disabilities in Minnesota's Basic Standards Exams (a copy of the survey is included in the Appendix). Questions were formatted to include closed response, open ended, and 5-point Likert scale responses. The survey offered respondents the opportunity to provide anecdotal information about written guidelines, specific accommodations used by each district, and future needs.
Fourteen districts reflecting the mix of urban, metropolitan, and rural settings returned surveys describing participation rates of students with disabilities in the Basic Standards Exams. Additionally, a few districts enclosed written policies that they used to guide their decision making process about the participation of students with disabilities in the recent assessment. All 14 districts reported that at least 50% of all eligible students with disabilities participated in the April 1996 testing cycle.
Ten districts indicated that they had written guidelines to assist in making decisions about including students with disabilities in the assessments. However, only six districts indicated that they had used them during the recent testing cycle. While ensuring compliance to a set of district guidelines may present a challenge during future assessments, an initial review of these district guidelines is extremely encouraging. That is, based on a recent publication (Elliott, Thurlow, & Ysseldyke, 1996) that identified core policies and guidelines that (a) promote the participation of students in assessments, and (b) define appropriate assessment accommodations, almost all aspects of these criteria were present within each district's written guidelines. For example, Elliott and her colleagues recommend that all students (including students with disabilities and LEP) participate in statewide assessments. One district's policy guidelines emphatically meet this guideline, stating in the introduction that "All students with disabilities will participate in the Graduation Standards. Most students with disabilities, when provided with appropriately designed learning opportunities, will participate successfully in the Graduation Standards assessment processes." Other core policies and guidelines include: (a) who should make participation decisions for students; (b) the use of forms to guide and document participation decisions; and (c) inclusion of families to receive information regarding these policies as well as potential implications these decisions will have on their child.
On the other hand, when districts did not have guidelines or chose not to use them, they often indicated that participation decisions were made on the basis of either disability category or severity of disability. For example, one district chose to exempt students with either moderate or severe mental impairments. Many districts also indicated that decisions were made on a case-by-case basis. Comments made by districts that did not have guidelines are shown in Table 1.
| TABLE 1: Sample of Comments by Districts Not Using Written Participation Guidelines |
| "We wanted baseline data so we decided to test
all students." "The only exemptions were by category (SMI, MMI)." "Decisions were based on a case-by-case basis with category or severity of disability as the indicator." "Every student with an IEP . . . was tested. The exceptions were students identified as moderately or severely mentally impaired." |
Of those districts with written guidelines, most indicated that the guidelines had been produced by more than one individual or by a specific group of individuals. Typically, district committees, local special education directors, and district testing coordinators were most frequently endorsed as having produced their district guidelines. When more than one set of guidelines was available to district personnel, guidelines produced by a district committee tended to be used most extensively. These committees were composed primarily of parents, teachers, district test coordinators, and administrators. Other districts indicated relying on the guidelines of the Department of Children, Families and Learning most extensively.
Although accommodations were allowed and guidelines on allowable accommodations were provided by the Department of Children, Families and Learning, some districts reported being confused about the process. For example, two districts stated that they thought accommodations were not allowed in this "pilot" year. Another district indicated that a plan for making accommodations was not in place at the time of the initial testing; thus, only the provided audiocassette presentation for the math test was used. On the positive side, nine districts indicated that they had specific written guidelines to expedite the allocation of accommodations to students with disabilities. These nine districts were then asked to rate the helpfulness of the written guidelines based on the degree to which they helped to:
District responses are presented in Table 2.
| TABLE 2: Number and Percentages of Districts Indicating Extent to Which Guidelines Were Helpful in Determining Participation of Students | |||||
1 Not Very Helpful |
2 |
3 |
4 |
5 Very Helpful |
|
| Determining whether particular students with disabilities should participate | 0 |
0 |
2 (22%) |
4 (44%) |
3 (33%) |
| Choosing specific accommodations for use when administering the test | 0 |
0 |
3 (33%) |
1 (11%) |
5 (56%) |
| Making modifications to the test, such as eliminating certain test items, or adjusting the performance standard | 0 |
0 |
4 (44%) |
2 (22%) |
3 (33%) |
N = 9
In general, districts found written guidelines helpful when determining whether to include students with disabilities, selecting appropriate accommodations, and making modifications for students with disabilities. Written guidelines appeared to be particularly helpful in choosing specific accommodations. Five of the respondents indicated that written guidelines were "very helpful" in choosing specific accommodations for use when administering the Basic Exams. Districts also indicated that written guidelines encouraged the participation of students with disabilities. Four of the respondents replied that guidelines encouraged participation "to a great extent" with a mean response across ten districts of 4 and a modal response of 5 on an "encouragement" scale of 1 to 5.
Ten of the responding districts provided information on the types of accommodations used during the testing period. Results from the survey indicate that the most common types of accommodations used for students with disabilities were timing/scheduling and presentation. As is evidenced in Table 3, 40% of the districts indicated that they had extended the time allotted to complete the test. Other timing/scheduling accommodations included allowing frequent breaks, administering the test in several sessions, and administering the test over several days. None of the districts indicated that they altered the time of day that the test is administered.
| TABLE 3: Timing/Scheduling Accommodations | |
| Extend the time allotted to complete the test | 4 (40%) |
| Administer test in several sessions over course of day | 1 (10%) |
| Administer test in several sessions over several days | 1 (10%) |
| Allow frequent breaks during testing | 1 (10%) |
N = 10
Reading the test aloud was the most common presentation accommodation (see Table 4). None of the districts indicated that large print, magnification, or Braille versions had been used by students with disabilities.
| TABLE 4: Presentation Accommodations | |
| Reading test aloud | 4 (40%) |
| Repeated directions | 2 (20%) |
| Sign lnaguage assistance | 2 (20%) |
| Audiocassette | 1 (10%) |
N = 10
Districts appeared to limit setting accommodations to either small group administrations (in 4 or 40% of cases) or separate room administrations (in 4 or 40% of cases). None of the districts indicated use of a study carrel or administration in a completely separate setting (e.g., hospital, homebound, home school). The only response accommodation that districts reported allowing was sign language assistance (in 2 or 20% of cases).
Three themes emerged from the responses districts made to the survey question about additional information they would find helpful. First, several respondents indicated that knowing the results of testing for students with disabilities would be helpful. As one respondent indicated, "It would be helpful to know how other students from other schools with the same disabilities scored compared to us."
Second, districts indicated a desire to have sample policies and information from other districts available to them. Districts also thought that guidelines and criteria established by the Department of Children, Families and Learning would be helpful in defining reasonable accommodations and in establishing who should help to make the decisions about student participation and inclusion within the testing process.
Finally, seven of the districts indicated that they had been involved in collaborative work with other districts to discuss and propose assessment policies for students with disabilities. All of these districts expressed high satisfaction with these experiences and recommended that the Department of Children, Families and Learning encourage common forums for further collaboration.
These initial findings from 14 representative districts within the State of Minnesota are encouraging. In addition, responses indicated areas of concern that will need to be addressed in the near future. On the positive side, overall participation rates of students with disabilities were high for the first round of basic skills testing. In general, responding districts seemed committed to establishing guidelines and procedures that will enhance participation rates for all students. It appears the shift toward greater accountability for documenting and monitoring the progress of students with disabilities is one that is being accepted by districts in the State of Minnesota. Furthermore, districts that had written guidelines perceived them as helpful in: (a) making participation decisions, (b) choosing accommodations, and (c) potentially making modifications to the test to meet individual student needs. It will be important in the future for districts to continue to evaluate the impact that these written guidelines have on participation rates and their ability to guide appropriate decisions. To summarize, the guidelines reviewed to date appear to meet stringent published criteria (Elliott et al., 1996). This potentially bodes well for the future, as the Department of Children, Families and Learning attempts to enhance uniform implementation of Basic Skills testing across districts and to minimize discrepancies for students who transfer between districts.
The results of this survey also indicate that providing accommodations is an area of concern and often inconsistently interpreted and implemented across districts. While it is not clear why certain accommodations were chosen by the districts, each particular respondent used a variety of accommodations for students with disabilities. Individual districts, along with the Department of Children, Families and Learning, will need to be aware of how accommodation decisions are made and what the outcomes of those accommodations are on a number of factors (e.g., reliability, validity, technical adequacy, referral rates, exemptions, etc.). Empirical evidence evaluating the impact and use of testing accommodations for students with disabilities is scarce to nonexistent. Yet districts are continuously being asked to make decisions that could have major life consequences for students with disabilities. The best way to examine the outcomes of accommodations is for districts and the Department of Children, Families and Learning to monitor student outcomes and to evaluate empirically the use of specific accommodations on student performance. Obviously, research is needed to answer many of the pressing technical and policy questions inherent in any large scale assessment process.
To summarize, districts continue to seek guidance and clear lines of communication from the Department of Children, Families and Learning. For the most part, districts are reporting initial compliance and support for a full inclusion policy of assessing all students, with "all" including students with disabilities. Additionally, each district seemed to indicate a need to "check in" with other districts as a way of pooling resources and building consensus across the state. Finally, there appears to be universal concern on how data will be summarized and shared within the State. It would appear that as important implementation decisions are made, maintaining a level of collaboration among districts and the Department of Children, Families and Learning would be a good strategy, one that could lead to more positive outcomes for students with disabilities as the State continues to plan for their participation in the Basic Standards Exams.
American Federation of Teachers. (1995, March). Making standards matter: A fifty state progress report on efforts to raise academic standards. Washington, DC: AFT.
Business Roundtable. (1996, June). A business leader's guide to setting academic standards. Washington, DC: Business Roundtable.
Education Commission of the States. (1996, March). Standards and education: A roadmap for state policymakers. Denver, CO: ECS.
Education Daily. (1996, July 23). "Work gets harder once states okay standards." Article by David Hoff (Volume 29, #141), pp. 1-2.
Elliott, J., Thurlow, M., & Ysseldyke, J. (1996). Assessment guidelines that maximize the participation of students with disabilities in large scale assessment: Characteristics and considerations. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.
McGrew, K.S., Thurlow, M.L., Shriner, J.G., & Spiegel, A.N. (1992, March). Inclusion of students with disabilities in national and state data collection programs. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.
Copy of Survey
To receive a copy of this appendix, please order a printed copy of the report from the Publications Office. Cost is $5.00.