Input from the Field on Assessing Students with Limited English Proficiency in Minnesota's Basic Requirements Exams

Minnesota Report 2

Published by the National Center on Educational Outcomes

Prepared by Richard Spicuzza, Ronald Erickson, Martha Thurlow, Kristin Liu, and Aaron Ruhland

August 1996

Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Spicuzza, R., Erickson, R., Thurlow, M., Liu, K., & Ruhland, A. (1996). Input from the field on assessing students with limited English proficiency in Minnesota's Basic Requirements Exams (Minnesota Report No. 2). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/MnReport2.html

Overview

Including students with Limited English Proficiency (LEP) in statewide assessment programs is a task embedded in challenges that include:

• The lack of consensus on the definition and classification of LEP students.
• The general lack of legal and empirical support to document the procedures for including LEP students in assessment.
• The delay of testing until students have a minimal level of English proficiency.
• The difficulty in making appropriate translations or decisions about the allowance of translations.
• The inconsistent record keeping on participation in the assessment process.

These are just some of the most basic challenges. Deeper issues surround questions about opportunity to learn (are students being tested on something that they have been taught?); the purpose of the assessment (is the test one of English ability or is it a test of knowledge?); how "minimum proficiency" is defined (can it be assessed separate from the influence of vocabulary knowledge?), and how the information will be used and the unintended consequences of that use (will students be encouraged not to attend school so that they do not bring down school scores?). Because of the complex interactions of language proficiency and knowledge, testing the knowledge of students who do not yet have completely developed language proficiency is a continuing challenge.

Advocates for students with limited English proficiency recommend collecting performance data on specific populations, developing the tests in translated forms, using temporary exemptions sparingly, using modifications cautiously, and using alternative assessments for LEP students who can't take other forms of the test (Rivera and Vincent 1996). Meeting these challenges is a part of including LEP students in the assessment of standards of learning.

Nationally, there is a strong push for higher standards of learning and the implementation of assessment programs to measure progress toward these higher standards (for example, see publications by the American Federation of Teachers, 1995; the Business Roundtable, 1996; the Education Commission of the States, 1996; and Education Daily, July 23). As states and schools implement these standards and assessments, they are faced with the challenge of ensuring that they include all students, particularly students with Limited English Proficiency and students with disabilities, who are excluded frequently from these types of educational initiatives (see for example, McGrew, Thurlow, Shriner, and Spiegel, 1992).

Currently, the State of Minnesota is facing the challenge of defining high standards, and to assess students' progress toward these articulated standards. In addition, policies and guidelines are being developed at a state and local level to encourage maximum participation of all children. Prior to 1994, there was no requirement in Minnesota for any kind of state level assessment. While individual school districts generally collected data in the past, the nature of the assessment varied from one district to another, and there was no attempt to provide a statewide picture of student performance for the public.

In response to federal and State legislation, Minnesota is currently developing an assessment system that will allow it to produce state-level reports on the performance of its students. The assessment system includes both Basic Standards Tests, designed to assess basic skills, and the Profile of Learning, designed to assess high-level instructional standards of learning. Together, this two-tiered approach is an effort to ensure that Minnesota students meet both basic skills requirements and challenging standards before graduating from high school.

The Basic Standards Tests for Reading and Mathematics were first implemented throughout the state on a voluntary basis during the 1995-96 school year. School districts were encouraged to participate in the assessment process, and to include all eligible students. The most recent April 1996 testing cycle was both legislated and conducted within a short-time span. Thus, relatively meager guidelines were provided to districts about how to make decisions about the participation of students in the exams or about possible accommodations that would encourage maximum participation of all students.

The recent implementation of the Basic Standards Tests within the state of Minnesota provided an opportunity for the Minnesota Assessment Project to examine: (a) overall participation rates of LEP students, (b) examples of written guidelines used to enhance the inclusion of LEP students in the Basic Tests, (c) retrospectively, how decisions were made to include or exclude LEP students, (d) what kinds of accommodations were made available, and (e) specific needs of districts to ensure they are able to include as many students as possible in these assessments.

To answer these questions a survey was distributed to 22 districts across the state of Minnesota. The survey was created in April by the University of Minnesota, in conjunction with the Minnesota Department of Children, Families and Learning (CFL). A subsequent survey was developed and distributed in May 1996 by the Department of CFL and sent to 165 districts. These data were perceived as being an important first step before gathering more in-depth information from district personnel on the participation of LEP students in the Basic Standards Tests.

Method

The initial surveys were distributed to 22 districts across the state of Minnesota. The districts were chosen to include a mix of urban, metropolitan, and rural settings. Each district had participated in the April 1996 administration of the Basic Standards Tests. The assessment coordinator in each of the 22 districts was contacted by phone to explain the purpose of the survey. Surveys were mailed to districts, and returned to the University of Minnesota for further analysis.

The survey consisted of eight questions about the participation of LEP students in Minnesota's Basic Standards Tests (a copy of the survey is included in Appendix A). Questions were formatted to include closed response, open ended, and 5-point Likert scale responses. The survey offered respondents the opportunity to provide anecdotal information about written guidelines, specific accommodations used by each district, and future needs.

The second survey was distributed by the Department of CFL to 165 districts across the state of Minnesota. These districts also represented a mix of urban, rural, and suburban districts. Surveys were sent by mail, returned to the CFL, and were forwarded to the University for analysis.

The second survey consisted of three questions about the participation rates of LEP students during the April 1996 testing cycle. (A copy of the survey is included in Appendix B.) These questions centered around written guidelines, respondents concerns, and respondents' recommendations for the participation of LEP students in the Basic Standards Tests. Questions were presented in open-ended and closed-response format. The Department of CFL also asked the respondents to include a copy of their written guidelines.

Findings

University of Minnesota Survey 1

Thirteen districts returned surveys describing the participation rates of LEP students in the Basic Standards Tests. Some districts included written guidelines used to guide their decisionmaking about the participation of LEP students. Ten of the responding districts reported that at least 50% of eligible LEP students participated in the exams; two districts reported that less than 50% of eligible LEP students participated; and one district had no LEP students during the April 1996 testing cycle.

Nine districts indicated that they had written guidelines to assist them in making decisions about including LEP students in the Basic Standards Tests. However, only five districts indicated that they had used them during the recent testing cycle. When districts did not have guidelines or chose not to use them, participation decisions were made on a case-by-case basis as recommended by the English as a Second Language (ESL) teacher, or were based on length of stay in the United States, the student's ability to communicate in English, or the degree to which the student was mainstreamed. Two districts indicated that all students were tested, one indicating that the rationale was to get baseline data. Comments made by districts that did not have guidelines are shown in Table 1.

Of the districts with written guidelines, most were produced by multiple sources. District committees produced the most extensively used guidelines among the responding districts. District committees were commonly comprised of parents, test coordinators, ESL teachers, and administrators. A district committee was involved in six of the nine districts' production of written guidelines.

Other primary contributors included district test coordinators, ESL educators, and the Department of CFL. Two respondents indicated input from special educators. Three indicated that the Department of CFL was partially responsible for helping to produce their written guidelines, and two of the districts followed the Department of CFL guidelines most extensively.

Helpfulness of Written Guidelines. Nine districts with written guidelines rated the helpfulness of the guidelines in making three types of decisions:

• Determining whether particular students with limited English proficiency should participate.
• Choosing specific accommodations for use when administering the test.
• Making modifications to the test, such as eliminating certain test items, or adjusting the performance standard.

A five-point Likert scale was used to rate district guidelines on each purpose. The number and percentage of districts indicating each level of helpfulness for each factor is shown in Table 2.

N = 9

Respondents indicated that written guidelines were very helpful in determining participants, choosing accommodations, and making modifications for LEP students. Seven each responded that guidelines were "very helpful" in determining participants and in choosing accommodations, and six responded that guidelines were "very helpful" in making modifications. Four respondents replied that guidelines encouraged participation "to a great extent" with a mean response of 3.9, and a mode response of 5 across nine districts on an "encouragement" scale of 1 to 5.

Accommodations Used. The most common types of accommodations made for LEP students were timing/scheduling accommodations, with the most common example being extended time to complete the test. Other timing/scheduling accommodations that were used by the respondents included altering the time of administration, allowing frequent breaks during testing, administering the test in several sessions, and administering the test in several sessions over several days. Timing/scheduling accommodations are identified by the percentage of responding districts who used them, in Table 3.

N = 9

It is important to remember that the denominator for calculating percentages is 9 districts; thus, 11% represents just one district. It is also important to remember that the frequency of actual use within a district could vary dramatically between school sites.

Other types of accommodations were not extensively implemented by the responding districts for LEP students. Setting accommodations were made by five districts who used small group administration and four who provided separate room administrations. None of the districts reported using homebound, home school, or hospital administration, and none reported using a study carrel for administration.

Two presentation accommodations were reported: reading the test aloud by two of the districts and repeating directions by three of the districts. No districts reported using audiocassettee versions of the test. Presentation and setting accommodations are identified by the percentage of responding districts who used them in Table 4.

N = 9

No response accommodations were acknowledged by any of the responding districts. Those accommodations include dictating to a scribe, sign language assistance, recording of answers, and having someone transfer answers from a booklet to an answer sheet. Also included in response accommodations are the use of Braille writers and word processors.

The accommodations that were acknowledged by the districts were the ones provided for LEP students during the April 1996 testing cycle. The responses to the survey did not indicate the reason behind why particular accommodations were used or not used or whether they would be allowed in future testing situations. Two districts indicated that accommodations were not allowed during this "pilot year." These responses were the only indication of why particular accommodations were chosen or exempted from the April 1996 testing cycle.

District Needs. Additional requested information centered around examples from other districts and clear guidelines from the Department of CFL. According to the respondents, those guidelines should address which accommodations are allowable for LEP students, the temporary three-year exemption window, what foreign language materials are available from the state, and what accommodations will be made for languages other than the translated ones. Finally, one respondent expressed interest in what types of remediation are suggested or required so that LEP students are successful in passing the exams.

Collaboration with other districts regarding LEP students was not an extensive practice; only four of the thirteen respondents noted such collaboration. Two of the respondents who acknowledged they had worked with other districts did so at a Department of CFL input session. The other two respondents collaborated with neighboring districts at regional meetings and workshops.

Department of CFL Survey 2

Forty-eight districts, representing a mix of urban, metropolitan, and rural settings, returned surveys. The majority of respondents included LEP students in the April 1996 testing cycle. The number and percentage of responding districts who included LEP students is listed in Table 5.

N = 47

Twenty-eight (58%) of the responding districts stated they had written guidelines regarding the inclusion of LEP students in the exams. Eighteen (38%) of the districts did not have guidelines in place at the time of testing, and of those districts, eight still included LEP students during the April 1996 Basic Standards testing cycle. One district indicated their guidelines were not finalized at the time this survey was completed.

Respondents' Written Guidelines. The written guidelines submitted by 26 districts were remarkably similar in content. The comprehensiveness of the district's guidelines varied, however, ranging from a paragraph describing LEP committee procedures to two pages of allowable modifications and accommodations, LEP policies, and procedures. Common themes included:

• Modifications and translations not being permitted on the Basic Standards Test in Reading.
• Language translations being allowed on the Basic Standards Test in Mathematics; and
• The existence of a three-year window for temporary exemption from the Tests.

The terminology used to explain the three-year exemption from testing for LEP students differed, and in some cases it was difficult to determine if the guidelines required the students to be in the district for three years or simply in a classroom where the primary language of instruction is English for at least three years. There was also some confusion in the written guidelines as to whether the Department of CFL provided the districts with translated versions of the mathematics test or if it was up to the district.

Some notable exemptions from many of the written guidelines were remediation policies for students who did not pass the Basic Tests and explanation of the "Pass-State", "Pass-Translate", and "Exempt" criteria (see Table 6.)

Five (19%) of the districts who submitted written guidelines mentioned remediation in those guidelines and one provided documentation of remediation procedures in their guidelines. Four (15%) of the districts who submitted written guidelines defined the terminology "Pass-State", "Pass-Translate", and "Exempt" in their guidelines.

Several policies within the districts' written guidelines contrasted the Department of CFL's written guidelines. In regard to exemptions, one district allowed exemptions from only the reading test, another district allowed exemptions for students who have been in the district more than three years, and one district allowed permanent exemptions. Another district based their exemption policy on the student's language proficiency, regardless of the time spent in a primarily English speaking classroom. One district did not allow an LEP student to receive a diploma until they had passed the Basic Standards tests.

Respondent Concerns. Responses to the open-ended questions varied greatly and were indicative of the lack of consensus on issues surrounding the participation of LEP students. There were also themes that more than one respondent indicated was an area of concern. The most frequently noted concern was how to interpret and use the temporary exemption from testing allowed for LEP students who have received less than three years of primarily English instruction. Respondents were concerned that students whose primary language of instruction was English for only three years would not have the necessary skills to read at eighth grade proficiency. One respondent suggested that "the 'field' knows that the amount of time necessary to be able to cognitively acclimate to a second language is seven years."

Other concerns included the lack of information about the tests (specifically writing), and the difficulty of providing remediation in a short period of time with limited funds. These districts requested more information from the Department of CFL as well as other districts; recommendations from the Department of CFL regarding inclusion of LEP students in the Basic Standards Tests; and funding to assist LEP students in meeting the Graduation Standards. One district thought the reading test was culturally biased against LEP and low income students.

In contrast to several other districts, one respondent indicated that they believed the use of the temporary exemption was not in the best interest of the students. This respondent replied, "LEP students may temporarily exempt themselves right out of the system." They planned on requiring a passing score on the exams in order to receive a diploma.

Only five respondents included questions for the Department of CFL regarding the testing of LEP students. Two of the questions posed by the respondents were related to accommodations. They centered around what accommodations were most appropriate or allowed, and if they would be noted on the diploma or transcript. Other questions related to what information and guidelines would be available from the state LEP office and/or other districts. One respondent was interested in how the writing tests will be analyzed and scored. Another respondent wanted to know what types of things would be tested so they could communicate this to students.

Respondent Recommendations. While few districts offered recommendations to the Department of CFL, those who did respond provided a variety of suggestions. One recommendation was to provide translations of the mathematics test in several languages. Another respondent believed that students having less then five years of English as the primary language of instruction should not have the same requirements as English speaking students. Two respondents thought that a bilingual translator should read the tests to ESL students. One recommended that LEP students receive extra help in preparing to meet the standards, and another respondent thought that the passing score should not exceed 75% correct.

Discussion

Results from the surveys indicate some encouraging findings along with a few areas of concern. Overall participation rates of LEP students were high for the first round of testing and the responding districts seem committed to establishing guidelines and procedures which will enhance those participation rates. In general, the shift towards greater accountability for the progress of LEP students is one which is being accepted by districts in Minnesota.

The districts who had completed written guidelines clearly perceived them as helpful in making participation decisions, choosing accommodations, and potentially making modifications to the test. Results from the second survey exemplify the disparity in guidelines between districts and strongly suggest that clear dissemination of LEP regulations are needed. The terminology and procedures within the Basic Standards Tests are still being interpreted and clarified by the districts and would benefit from further assistance by the Department of CFL. It will be important in the future for districts to be consistent with their written guidelines to provide uniform implementation across districts and to minimize discrepancies between districts.

The results of these surveys also indicate that providing accommodations is an area of concern for the districts. While it is not clear why certain accommodations were chosen by the districts, each particular respondent used a variety of accommodations for LEP students. Individual districts, along with the Department of CFL, will need to be aware of how accommodation decisions are made and what the outcomes of those accommodations are. The empirical evidence supporting accommodations for LEP students in a testing environment is scarce and should be researched. The best way to examine the outcomes of accommodations is for districts and the CFL to track student outcomes and to decide upon valid and allowable accommodations.

Districts were concerned about the "Exempt" status for LEP students. Some districts thought the three year exemption would allow students to be exempt from testing altogether. Others thought that the window should be extended beyond three years due to the time it takes to acclimate to a second language. Finally, a third group contends the partial/temporary exemption may need to be different by content area due to interaction of language acquisition and performance. Other concerns regarding provision of services to LEP students will need to be addressed with perhaps a more organized statewide system of classification, tracking, and placement to aid districts who do not have an abundance of resources.

To summarize, many of the districts who responded to these surveys were in the initial stages of producing or revising written guidelines for testing LEP students in Minnesota's Basic Standards Tests. Districts appear to be asking for even more specific guidelines or concrete examples from the Department of CFL. There appears to be a strong need for clarification on the issues of exempting students, translating the mathematics test, and providing remedial services to LEP students. Further collaboration on these issues would seem to be a good starting point in developing statewide guidelines that encourage the participation and success of LEP students in Minnesota's Basic Standards Tests.

References

American Federation of Teachers. (1995, March). Making standards matter: A fifty state progress report on efforts to raise academic standards. Washington, DC: AFT.

Business Roundtable. (1996, June). A business leader's guide to setting academic standards. Washington, DC: Business Roundtable.

Education Commission of the States. (1996, March). Standards and education: A roadmap for state policymakers. Denver, CO: ECS.

Education Daily. (1996, July 23). "Work gets harder once states okay standards." Article by David Hoff (Volume 29, #141), pp. 1-2.

McGrew, K.S., Thurlow, M.L., Shriner, J.G., & Spiegel, A.N. (1992, March). Inclusion of students with disabilities in national and state data collection programs. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Rivera, C. and Vincent, C. (1996). High school graduation testing: Policies and practices in the assessment of limited English proficient students. Paper presented at the Council of Chief State School Officers, Phoenix. Arlington, VA: Center for Equity and Excellence in Education.

Appendices

Copy of Survey Distributed by Project Staff at University of Minnesota

Copy of Survey Distributed by Project Advisors from Minnesota Department of Children, Families and Learning

For copies of these appendices, please order a printed copy of the report from the Publications Office. Cost is $5.00.

Top of page