Educators' Responses to LEP Students' Participation in the 1997 Basic Standards Testing

Minnesota Report 15

Published by the National Center on Educational Outcomes

Prepared by Kristen Liu, Richard Spicuzza, Ron Erickson, Martha Thurlow, and Aaron Ruhland

October 1997

Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Liu, K., Spicuzza, R., Erickson, R., Thurlow M., & Ruhland, A. (1997). Educator's responses to LEP students' participation in the 1997 Basic Standards testing (Minnesota Report No. 15). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/MnReport15.html

Introduction

The majority of the states in the United States have at least one statewide assessment. While the tests are said to be for all students, August and Lara (1996, cited in National Research Council, 1997) found that only five states require limited English proficient (LEP) students to take these exams. Furthermore, three of those five states allow LEP students to be exempted in certain circumstances. Of the 36 states that make a practice of exempting LEP students, 22 allow exemptions only on a temporary basis while the student learns English and becomes more acculturated to the United States.

While exemptions for LEP students may seem reasonable, researchers and educators are concerned that exemptions actually harm the students by allowing their educational needs to remain unaddressed. Students who are not assessed tend not to receive much needed educational services. For this reason, there is general consensus that LEP students must be included in statewide exams to the greatest extent that is practical.

There is some evidence that greater numbers of LEP students could be included. Researchers for the National Assessment of Educational Progress (NAEP) found that many LEP students who were excluded from NAEP could actually have participated in the test (National Academy of Education, 1996). The researchers recommended developing appropriate accommodations and modifications to enable more LEP students to participate in NAEP. This recommendation has value for many statewide exams as well, as states struggle to increase the number of LEP students who participate. In particular, the recommendation is relevant to assessments similar in purpose to NAEP, that is, designed for general accountability purposes.

Minnesota is one state that has developed a statewide assessment in response to the national movement toward state accountability and high standards for all students. The Minnesota Basic Standards Tests currently assess whether a student has achieved a minimum level of competency in reading, math, and writing. The Basic Standards Tests are high stakes tests that a student is required to pass in order to receive a diploma. LEP students in Minnesota are allowed to have a temporary exemption from the tests if they have been in an English speaking school for less than three years. Currently there are other testing considerations and modifications for LEP students, including translation, audio cassettes of the math test, and small group testing. These considerations and accommodations were developed to try to increase LEP student participation in the assessment.

The first round of Basic Standards Tests was voluntary and occurred in April 1996. The second round of testing, in January 1997, was mandatory for all districts that chose to use the statewide assessment. (Districts had the option of selecting another test, but had to equate it to the state test for determining a passing score.) Data showing how many LEP students were exempted from the 1997 tests are not yet available, but a survey of districts participating in the voluntary round of testing (in 1996) indicated that the majority of those districts tested more than 50% of the LEP students enrolled in the districts.

The Minnesota Department of Children, Families and Learning (CFL) and the National Center on Educational Outcomes (NCEO) encourage districts to include even greater numbers of LEP students in the testing so that the total picture of their educational needs becomes known and is addressed. Efforts to increase participation rates need to be based on knowledge of what actually happened in the first mandatory round of testing. In order to gain knowledge about what actually happened, and to examine whether more changes to the testing process are needed to include more LEP students, the Minnesota Assessment Project gathered data on the following questions:

• Who made participation and testing consideration/accommodation decisions for LEP students?

• What criteria were used to guide decision making?

• When were participation and testing consideration/accommodation decisions made?

• What testing considerations/accommodations were allowed in individual districts?

• What considerations/accommodations not currently mentioned in policy are desirable for LEP students and would increase their participation in the Basic Standards Tests?

• Are there teaching strategies educators use with LEP students in the classroom that may be helpful on the Basic Standards Tests?

Method

To answer the questions listed above, a survey was developed for educators and administrators to complete about LEP students in the 1997 Basic Standards Tests. Survey response formats included closed response, open-ended response, and checklists with rank ordering of checked items (see Appendix A).

In April and May of 1997, a total of 204 surveys were sent to 60 schools in 22 districts across the state of Minnesota. The 22 districts were chosen from a larger set of 28 districts that had previously agreed to work with the CFL during the development and administration of the Basic Standards Tests. The 22 were those with relatively large English as a Second Language (ESL)/ Bilingual Education programs.

The CFL classifies all districts in the state of Minnesota into one of four categories based on size and location. The categories are:

• Cities of the First Class (large, urban school districts).

• Suburban-Metro.

• Greater Minnesota > 2000 (rural districts with more than 2,000 students).

• Greater Minnesota < 2000 (rural districts with fewer than 2,000 students).

The classifications of the 22 districts included in this study are included in Table 1. Three districts were in cities of the first class, seven were in greater Minnesota with a population over 2,000, five were in greater Minnesota with a population under 2,000, and seven were in suburban-metro areas.

After districts were chosen, individual schools were identified that had both an ESL or Bilingual Education program and LEP students in grades 8 and 9 (the grades in which the tests are first administered.) Schools that had LEP students only in 10th grade or above were not included, and schools that did not have an ESL/bilingual program were not included.

Each school that was identified as having both an ESL/Bilingual Education program and either 8th or 9th grade LEP students received surveys; however, the number of surveys sent to each school varied depending on the size of the program. In each case, the principal or a contact person in the school received one survey for each ESL/Bilingual Education teacher in the building plus one extra survey for making copies. If a teacher worked in different schools within the district, the teacher was asked to fill out the survey only once, but he or she could include information from all of the schools. The principal or contact person was asked to distribute the surveys to those educators who were the most knowledgeable about the participation of LEP students in the 1997 Basic Standards Tests, and to make as many copies of the survey as needed. After surveys were completed, the principal or contact person was asked to collect the surveys and send them to NCEO for analysis.

This method of disseminating surveys was chosen because some schools include ESL/Bilingual Education teachers in the testing process, and others do not. Having the school contact decide who should receive surveys allowed us to target respondents with the most knowledge about the participation of LEP students. Additionally, we believed that if the principal was involved in the survey and stressed the importance of participating, more surveys would be returned.

Once surveys were returned, they were numbered and all quantitative responses were entered into a computer database. The qualitative responses were highlighted and compiled into a large list that was later organized into groups of related responses.

Table 1. Return Rates of LEP Surveys by District

Findings

A total of 96 surveys (47%) were returned from 45 of the 60 schools (75%). The individual return rate of 47% probably has little meaning since extra surveys were sent to each school and contacts had the option to give out either fewer surveys than we sent or more surveys than we sent. Because some of the returned surveys were completed by groups of people (e.g., 3 ESL teachers together), this report often talks about the number of responses instead of the number of respondents. Surveys filled out by multiple respondents were counted as one response unless there were multiple sets of answers. Table 1 shows the number of responses received from each district.

Respondents were primarily ESL/Bilingual educators, but other educators and administrators participated as well. Table 2 shows the breakdown of respondents by position. Data from surveys were both quantitative and qualitative. The quantitative data are addressed first.

Participation Decisions

When asked about the decision-making process used to determine whether LEP students should participate in the Basic Skills Tests, 58% (51 of 88 responses) indicated it was adequate. Another 33% of responses (29 of 88) came from educators who did not think that the decision-making process was adequate. Seven responses indicated that respondents did not know whether the process was adequate, and one response did not answer this item. When viewed by district size, 86% of responses in the “inadequate process” category and 100% of the responses in the “do not know what the process is” category came from educators in large, urban districts. When viewed according to the respondent's position, 100% of the principals (2 out of 2) and 89% of the counselors (8 out of 9) reported that they thought the process was adequate. In contrast, 48% of the responses from ESL/Bilingual educators (29 of 60) reported that the decision-making process was adequate, and 42% (25 of 60) reported that the process was not adequate.

Table 2. Positions Held by Survey Respondents

Overall, 36% of the responses (32 of 90) indicated that participation decisions for LEP students were made before January 1st, and 32% (29 of 90) indicated that decisions were made after January 1st and just a few weeks prior to the test administration. Twenty-six percent of the responses said that the respondents did not know when participation decisions were made. Again, when viewed by district type, 91% of responses that said respondents did not know when decisions were made (21 of 23) came from large, urban districts. Finally, 1%, or one response, did not have an answer to this item.

When asked who made the participation decisions, 42% of the responses (39 of 92) stated that some type of committee had made the decision. However, of these responses, only 8% (3 of 39) said that the LEP student was a part of the committee, and only 15% (6 of 39) said that the parent of an LEP student (not necessarily the parent of the individual student) was on the committee. The second and third most common responses after “committee decision” were “I do not know who made decisions” with 32% (29 of 92) of responses falling in this category, and “individual teacher decision” with 14% of responses (13 of 92) falling in this category. Again, 1%, or one response, did not have an answer for this item. When broken down by district size, 93% of responses indicating a lack of knowledge about who made participation decisions for LEP students came from large, urban districts.

Testing Considerations and Accommodations

Process. When asked about the process for making decisions about testing considerations and accommodations, 48% of responses (44 of 91) indicated that the process was adequate, 35% (32 of 91) indicated that the process was not adequate, 14% (13 of 91) indicated that they did not know what the process was, and 1% (1 of 91) had no answer for this item. Broken down by district type, 88% (28 of 32 responses) in the inadequate category came from large, urban districts. In addition, 92% (12 of 13 responses) indicating that respondents were unaware of what the decision-making process was also came from large urban districts. In contrast, 100% (18 of 18 responses) from rural districts of more than 2,000 students indicated satisfaction with the decision-making process. Viewed according to the respondents' positions, 88% (28 of 32) indicating that the decision-making process was inadequate were ESL/Bilingual educators and 95% (12 of 13) of responses indicating a lack of knowledge about the process were also from ESL/Bilingual educators.

Overall, 30% of responses (26 of 88) stated that decisions relating to testing considerations and accommodations were made before January 1st, and 25% (22 of 88) indicated that decisions were made after January 1st and just a few weeks prior to testing. Thirty-six percent of responses (32 of 88) said that the respondents did not know when the decisions had been made, and 6% (5 of 88) did not have an answer for this item. Further breakdown of the data shows that 100% (32 of 32) of responses demonstrating a lack of awareness about when testing decisions took place came from large urban districts.

Decision makers. When asked who made decisions about testing considerations and accommodations, 42% of responses (37 of 88) said a committee made them, 33% (29 of 88) indicated that respondents did not know who made them, 9% (8 of 88) said that an individual teacher made them, and 2% (2 of 88) did not have an answer for the item. Of the 42% that stated decisions were made by a committee, only 5% (2 of 37) said that the LEP student in question was a part of the committee, and only 11% (4 of 37) said that a parent of an LEP student was on the committee. Again, when the data were broken down by district size, 96% of the responses (28 of 29) saying that respondents did not know who made the decisions came from large, urban districts.

How decisions were made about testing considerations/accommodations. One item (2b) asked respondents to write a sentence describing how considerations/accommodations decisions were made. Answers to this question varied greatly, suggesting perhaps that the point of the question was not clear. Many responses focused on people involved in decision making. Several responses addressed more than one aspect of the question and even digressed to other topics. The number of times that specific points were mentioned were as follows:

• I don't know how decisions were made — 22 times

• The same decision was made for all LEP students — 13 times:

— no accommodations were allowed for any LEP students — 11 times

— the same accommodation was given to all LEP students — 1 time

— the type of decision was not specified — 1 time

• A parent/parents were allowed to choose — 4 times

• The student was presented with options and allowed to choose — 4 times

• The ESL teacher looked at individual student needs (e.g., emotional, academic, English fluency) — 3 times

• Written guidelines were referred to — 2 times

• The LEP contact person at the CFL was consulted — 1 time

• Input from the LEP community was obtained — 2 times

• Students and parents together were allowed to choose — 1 time

When these comments were categorized by district type, all of the “I don't' know” responses came from respondents in large, urban school districts.

Testing Considerations and Accommodations used in 1997

The third part of the survey asked respondents to tell which of the testing considerations and accommodations currently allowed were used by LEP students in the 1997 Basic Standards Tests, and roughly what percentage of students used each one. Only a small number of responses included percentages for each consideration or accommodation. The majority of responses simply indicated whether an accommodation had been used.

Table 3. Number of Responses Indicating that a Consideration or Accommodation was Given to LEP Students

Note: Entries in table are numbers of responses indicating that the accommodation or consideration was given to LEP students.

* These testing options result in a modified standard of passing.

Table 3 shows that the most frequently used testing consideration for both the reading and math tests was extra time. This was true even though the test was not timed. On the reading test, both individual or small group testing and standard testing using no accommodations were the second most commonly reported accommodations or options used. The second most commonly used accommodation on the math test was clarification or translation of the script. A few responses gave anecdotal information that suggested translation of test directions only happened for students from the predominant language group within the school and that students from other language groups did not receive the accommodation.

Desired Testing Options

The fourth section of the survey asked respondents to look at a list of possible testing options not currently mentioned in policy and to indicate which of the options they would like to have available for LEP students. Originally, respondents were asked to rank order their top three choices from most desirable to least desirable. However, because a significant proportion of responses did not provide rankings, we can only report which options or accommodations were chosen most frequently as desirable ones, not which ones were ranked highest (see Table 4).

Respondents chose the use of a familiar examiner as the most desired testing option or accommodation. This option is currently allowed, but is not mentioned in testing policy documents. The second most desirable accommodation option was the use of bilingual dictionaries or electronic translators. Anecdotal comments on the surveys pointed out that respondents preferred the use of bilingual dictionaries because LEP students use dictionaries in regular instruction and may continue to use them throughout their lives. The third most desirable option or accommodation was shorter versions of the reading and math tests. Also mentioned on more than half of the responses were simplifying the English on the entire test (56%), simplifying the English in the test directions (54%), and providing clarifying information at the end of the test booklet (54%).

Table 4. Testing Options Not Currently Mentioned in Policy that are Favored by Educators Who Work with LEP Students

Qualitative Results

In addition to the quantitative results discussed above, there were approximately 300 comments on various topics related to the Basic Standards Tests. Some of these comments were in answer to open-ended questions and others were written in the margins of the surveys. We compiled a master list of these comments, which was then separated into the following topic categories: (1) test validity and test bias; (2) participation decisions; (3) recommended accommodations and testing options; (4) accommodations decisions in 1997; (5) test preparation strategies; (6) needs; and (7) miscellaneous. The miscellaneous comments were randomly distributed throughout the six categories and title names were taken off of the lists.

To examine inter-rater reliability, two research assistants were given identical copies of the lists and were asked to do two tasks. First, they were to read through each list of comments and think of a broad category name for the list. Second, they were to circle any comments that did not seem to fit with this category name and suggest a more appropriate placement for each comment. We then compared the original list of categorized comments with the suggested categories provided by the two research assistants. If both research assistants made the same changes to the original list, those changes were incorporated when a final version of the list was created.

Overall, the same seven topic categories were identified in the final version. Major themes of each category are mentioned below. (For a complete list of comments, see Appendix B.)

Test validity and test bias. Responses indicated that respondents were concerned about the use of culturally biased reading material on the reading test and the frequent occurrence of slang or idiomatic speech. Unfamiliar idioms in a reading passage may significantly affect an LEP student's responses to test questions. In addition, there was concern over the amount of English that was involved in the math portion of the test. LEP students may be able to do the required math, but may not understand the language used in the word problems.

Participation decisions. Currently, participation decisions are based on the amount of time an LEP student has spent in English-speaking schools. The majority of responses stated that in addition to this (or in place of it) the student's English proficiency should be considered when making participation decisions. Some responses indicated that some districts had made the decision to test every LEP student without consideration of individual circumstances. The majority of these responses indicated dissatisfaction with the district decision.

Recommended testing options and accommodations. Many responses noted the need for bilingual dictionaries to be allowed as an accommodation. Responses also called for the test administrator to model problems using visuals (e.g., overhead transparencies of problems) just prior to testing. More sample problems and more visuals within the test booklet itself were desired as well. To complement these suggestions, respondents would like the test administrator to be able to clarify directions in ways that are different from the script that is provided.

Accommodations decisions in 1997. A number of responses mentioned that LEP students in some schools were not allowed to have any accommodations because baseline data on LEP student performance were needed. Accommodations were thought to alter a student's true performance. A few responses said that some students who would have been exempted from the test chose to take it anyway, and the people making testing decisions thought that such students did not need any accommodations. Some responses highlighted the frustration that educators felt when accommodations decisions were made by people who did not understand the needs of second language learners. A few respondents indicated their decisions had been reversed, including those that determined who would participate, and which accommodations were to be provided.

Test preparation strategies. A majority of responses with comments on this issue underscored the need for more sample tests and test preparation materials. Respondents wanted students to be instructed in how to take standardized multiple-choice tests and how to use different reading strategies. Some wanted students to be taught the meanings of words frequently used on tests (e.g., refers to, all except) and mathematics terms.

Needs. Comments about needs varied a great deal. However, several responses mentioned the need for more accurate demographic data on LEP students. Such data might include the date of arrival in the United States and the student's educational history. A standardized definition of the term “limited English proficient” was also desirable because it would make data collection easier. Respondents believed these pieces of information would aid schools in making more appropriate participation and accommodation decisions.

Miscellaneous. There were no common themes in this category. Comments ranged from “It isn't lack of trying, intelligence or preparation. They [LEP students] haven't had the time (5-7 years) to achieve academic competence in English” to “Too many students were confused, misinformed, frustrated, distressed and embarrassed by the whole process.”

Respondents' Recommendations

The following recommendations were made for improving the participation and performance of LEP students in the Basic Standards Tests:

1. Examine the test for biased content and language. In addition, look at the amount of English that is used on the math test. The math test may test English knowledge more than it tests mathematical concepts.

2. Investigate the impact of bilingual dictionaries or glossaries on LEP student performance. Because students use dictionaries in daily instruction and may continue to use dictionaries throughout their lives, the use of glossaries or dictionaries would be a reasonable accommodation.

3. Include a measure of English proficiency as one of the criteria for participation in the tests. Time spent in English-speaking schools is not a meaningful measure of English ability.

4. Develop special test preparation classes that teach test-taking skills and reading strategies.

5. Work on ways to improve the accuracy and detail of information in student records. This information can aid in making better participation and accommodation decisions.

6. Do not make the same accommodation decisions for all LEP students. Students are individuals with different needs and not all of them will be helped by having the same accommodation.

Discussion

The overall findings from this survey are beneficial because they expand our limited knowledge base on the assessment of LEP students. Additionally, the findings allow the Minnesota Department of Children, Families and Learning to begin to evaluate the impact of the test for these students. Results suggest both encouraging areas and areas that are of concern. First, it is encouraging that educators and administrators who work with LEP students are giving serious thought to the needs of the students when the students take the Basic Standards Tests. In general, the majority of responses showed that respondents believed the participation decision-making process used in their districts and schools was adequate. Also, just under half of the responses showed that respondents thought the process of determining accommodations and other testing considerations for LEP students was adequate. Unlike mandated team-based decision making for students with disabilities, in 1996-97 there was no requirement that teams of educators make decisions for LEP students. Yet, a significant proportion of responses said that some type of committee had, in fact, made participation and accommodation decisions for LEP students. Roughly one-third of responses indicated that participation decisions were made more than one month before the test administration date and nearly the same proportion indicated that accommodation decisions also were made more than one month before the test administration date.

At the same time, the survey responses clearly highlighted several major areas of concern that need to be addressed:

1. In spite of the fact that ESL/Bilingual teachers and bilingual educational assistants may be the educators with the most knowledge about the needs of LEP students, in large, urban districts these people are often unaware of, and uninvolved in, the process of making assessment decisions for LEP students. This lack of involvement is creating high levels of frustration among them.

2. The lack of information flow in large, urban districts is preventing educators from obtaining the knowledge they need to be a part of the decision-making process. Despite repeated training efforts by the CFL and the dissemination of testing accommodations documents, some responses showed that there are ESL/Bilingual educators who are unaware of the accommodations allowed for LEP students.

3. The original versions of testing accommodations documents did not specifically discuss all of the accommodations that are allowed. For example, on the survey “use of a familiar testing administrator” was chosen as the most desired accommodation not currently mentioned in policy. Few respondents were aware that this accommodation is allowable, probably because it was not mentioned in early versions of testing documents. In addition, early versions of the documents did not list short segment tests as an option for LEP students, even though they are allowed.

4. There is a general lack of understanding among educators and administrators as to the purpose of an accommodation. The CFL provided lists of allowable accommodations that may make tests more accessible for certain LEP students, but do not change the rigor of the testing situation. However, in 1997 some districts did not allow LEP students to have any accommodations in the belief that the accommodations would not allow a student's true performance to be seen, or in the belief that students did not need any accommodations.

5. Despite the fact that the test was not timed, responses frequently mentioned a concern about the lack of time. This implies an unspoken time limit that may arise because of peer pressure in a large group testing situation. Additionally, in focus group interviews with LEP students and parents (Quest, Liu & Thurlow, 1997), some students mentioned that they had been told to stop taking the test at a specific time so that they would not disrupt the school schedule. Because the test is a high-stakes assessment required for graduation, schools need to make allowances in scheduling for students who require extra time. CFL testing documents specifically state that students who will need more time should be tested in a separate location so that they do not feel pressure from peers who finish more quickly.

6. Differences of opinion and different priorities among administrators, testing coordinators, and educators who work with LEP students are a major source of frustration for ESL and Bilingual teachers. Because the surveys were distributed and collected by building principals, many respondents may not have voiced their feelings on this subject. However, those responses that did mention the issue spoke of administrators and testing coordinators who overrode decisions made by ESL/Bilingual educators and made decisions based on priorities other than the students' needs. Clearly, there must be more communication about the different issues involved in large-scale assessment. A decision-making tree with specific criteria should be established in each district and building.

7. A significant number of participation and accommodation decisions were made just prior to the date of testing. If LEP students, parents, and ESL and Bilingual educators are to be involved in the decision making, and if individual student needs are to be considered, this process should begin earlier so that there is time for dialogue among all parties involved.

These areas of concern, of course, are relevant not just to Minnesota's Basic Standards Testing. They reflect broader issues relating to the involvement of students with limited English proficiency in most large-scale assessments. As evidence that these kinds of concerns can be addressed, Minnesota's CFL responded to each of the six recommendations derived from the educators' responses. The extent to which states and districts are able to consider recommendations from the field and other areas of concern will affect the participation of LEP students in these assessments.

References

National Academy of Education. (1996). Quality and Utility: The 1994 TSA in reading. The fourth report of the NAE panel on evaluation of the NAEP TSA: 1994 TSA in reading. Stanford, CA: National Academy of Education, Stanford University, School of Education, 53-73.

National Research Council. (1997). Improving schools for language minority children: A research agenda. In D. August, & K. Hakuta (eds.). Washington, DC: National Academy Press.

Quest, C., Liu, K., & Thurlow, M. (1997). Cambodian, Hmong, Lao, Spanish-speaking, and Vietnamese parents and students speak out on Minnesota's Basic Standards Tests (Minnesota Report 12). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Appendix A

Survey on LEP Student Participation in Basic Standards Tests

Minnesota Assessment Project

Survey on LEP student participation in Basic Standards Tests

Name________________________________________________________________________________

School:___________________________________ District:____________________________________

What kind of position do you have? (circle one)

(a) principal

(b) counselor

(c) teacher working primarily with LEP students

(d) classroom teacher who has some LEP students

(e) other (please describe):_______________________________________

How many 8th and 9th grade LEP students do you work with? _________________________________

Participation

1a. Was the process used within your district or school adequate for determining whether LEP students should participate in the Basic Standards Testing? _____ Yes _____ No

What changes to the decision process would you recommend?

_____________________________________________________________________________

_____________________________________________________________________________

1b. For the recent testing cycle, when (i.e., time of year) were participation decisions made for LEP students?

1c. For the recent testing cycle, who made participation decisions about LEP students?

Testing Considerations & Accommodations

2a. Do you think the process used to determine testing considerations and accommodations for LEP students was adequate? (this process is included currently in the state guidelines)

_____ Yes _____ No

2b. How were decisions made about which accommodations were appropriate for LEP students?

2c. When (i.e., time of year) were these decisions made?

2d. Who decided which considerations and accommodations were appropriate for LEP students?

2e. Below is a table showing various ways that your LEP students may have participated in the recent Basic Standards Tests of Reading and Mathematics. For each option listed, estimate the percentage of students you teach who used the following testing considerations and accommodations (e.g. if approximately 3/6 students used extended time for the reading test, write "50%" in the box labeled "reading test").

If any of these considerations or accommodations were not used, please put a 0 in the appropriate box.

*These testing options result in a "Pass Translate" designation. The other options result in "Pass State."

3a. This fall, the Department of CFL conducted focus groups with LEP students and their parents to ask them for input on possible testing options. We would like to have input on options from teachers as well. Which of the following testing options would you like to have available for LEP students? Your suggestions will be shared with policymakers at the CFL and may influence future policy decisions. Please:

i) in the "I would like this" column, mark all that you would like with an X.

ii)in the "3 most desirable" column, select three items that are the most desirable and rank these top three options in order of desirability (1 = most desirable, 3 = third most desirable)

3b. Please list and describe any other strategies that educators currently use with LEP students during typical instruction that may be helpful to these students on the Basic Standards Tests.

THANK YOU FOR YOUR ASSISTANCE. PLEASE RETURN THIS COMPLETED FORM TO YOUR PRINCIPAL AND IT WILL BE FORWARDED TO THE UNIVERSITY OF MINNESOTA FOR ANALYSIS.

IF YOU HAVE ANY QUESTIONS, PLEASE CALL KRISTI LIU (UNIVERSITY OF MINNESOTA) AT 626-0546

Appendix B

Comments from Survey Respondents

1. Comments on Test Validity and Test Bias

Language in Tests

1. Use of articles/stories with less idiomatic/colloquial English (is recommended).

2. The use of newspaper articles often written with colloquial speech are really unfair to LEP students who are having a difficult time trying the literal meanings.

3. Do not use American idioms. This test is extremely difficult for second language learners. They need years to learn English before they are expected to pass the test.

4. Test is difficult for bilingual and bicultural students. It contains language that is hard for second language learners; i.e. idioms, and white, middle-class assumptions.

5. LEP students need not to be dealt (with) as the mainstream students when it comes to language.

6. As a second language speaker myself, it is inappropriate to use informal speech and slang words when we struggle hard enough to know the literal meanings of English words.

7. Newspaper articles are difficult enough (literal) let alone figurative meanings, ex: "chip on my shoulder."

8. Different reading passages should be used where the English use is Standard, not slang, idioms, colloquial (English)- especially as is found in personal interest stories.

9. Also, eliminate slang, idiomatic language, and stylistic newspaper jargon.

10. Using reading samples with commonly used vocabulary. My real concern is not so much with the reading level as with the unfamiliar vocabulary/subject matter.

11. Simplify the vocabulary.

12. Using basic vocabulary - simplified text.

13. During typical instruction, language is (usually) simplified and vocabulary is explained.

14. (We use) more simplified reading book (in class).

15. Use simple English.

Cultural Bias

16. I don't feel the design of the test questions themselves even remotely considered the needs of second language learners.

17. Tests should be modified to eliminate cultural bias.

18. We need to make sure reading selections are not culturally biased (They were this time).

19. I would suggest keeping news articles that don't deal with universal topics off the test, since it culturally biases the whole instrument.

20. Eliminate culturally biased readings (wild rice and Twinkies, for example would often not be understood by second language students). These characteristics make the readings unfairly difficult for second language learners.

21. Include stores that LEP students as well as regular students can relate to.

22. How about creating a fairer, less biased, less white middle class (Twinkies, Little League) test? We've (as an educational community) studied test bias for 30 years.

23. We are not allowed to look at the test so we have no idea about the cultural appropriateness for our students.

Math Test

24. Be very careful of appearance- especially on math- which is really a reading test, not one of math skills.

25. The language part of the math test could be simplified so it's more a math test than a reading test.

26. Math test was more English than math.

27. Math should have less story problems. It is testing comprehension rather than math skills.

2. Comments on Participation Decisions

1. I would have LEP students flagged for at least five years instead of three. They need more time to be fluent in their language skills.

2. 3 years is not enough time. They are NOT ready to take this test, and are allowed NO special adaptations at the moment to help them. Also, it is up to the ESL director in the school to identify these students.

3. I probably would not ask a student to take the test unless they have been in the country and school for at least two years. Even then they would not pass it, but it would give the district a baseline.

4. We were told that anyone in the US less than 3 years was exempt, and that all others must test. We should be given more leeway based on individual circumstances.

5. Students in the US more than 3 years (our standard) should be allowed to take the test; however, for some learners, that's like testing a 2nd grade student.

6. That new students not be tested- unless the data would be to indicate continued progress instead of determining graduation.

7. New comers should not be tested.

8. It's not very appropriate for new students to this country.

9. Beginners should not be subjected to this torture.

10. All ESL students took the test even if they had come only the.. (illegible)

11. Everyone was encouraged to be tested to collect data. Some are ready to try test after being here a year (instead of??) waiting for 3 years.

12. Basic Standards Testing is not appropriate to a new American student who just came to country.

13. LEP students that have been in the USA less than 3 years should not take the test. Students should have English proficiency levels 3, 4 at least, to be allowed to take the test.

14. (We made decisions) Based upon number of years in this country and primarily, ability to understand English.

15. All students in the country under 3 years were exempt, if they chose.

16. I made the decision based on the individual's proficiency level.

17. They should be tested when they do understand what they are doing, means learn English language.

18. The only process used is whether they are beyond zero English skills. I'd recommend intermediate (proficiency) with a year in USA minimal.

19. Extend 3 year cut off for modifications.

20. The students with English Language Proficiency Level III-IV should take the test.

21. The # of years in the USA should be considered and the level of the English skill should be part of the decision.

22. The students who don't speak English or have very limited English proficiency should not be subjected to such tests.

23. Intermediate students who haven't been in the country for 3 years also be allowed to take the test for practice, exposure and an understanding at what level needs to be attained and to establish a baseline to measure progress from.

24. I am firmly against special help for LEP students. The must learn all aspects of English to demonstrate proficiency. If everyone is accorded the same special considerations, they can be too. However, determining at what point the test is taken is another story.

25. Decisions (participation) were based on students' fluency level as determined by ESL teacher.

26. I was informed in the Fall of 1997 that any of my 8th grade LEP students who had been in country, or in US/English speaking school for three years or less were exempt from taking the state test. This holds true for other tests as well. I was asked at that time for a list of those students' names.

27. Students in country less than 3 years did not have to take the test. District testing office made this decision.

28. At our school, I made the decision (teacher). I also conferred with the counseling staff.

29. All students should be referred by ESL teacher as to who should take tests. If in doubt, let them do it to have an idea.

30. The teachers, parents, students, and bilingual staff (made participation decisions).

31. The decision for our school was made by our LEP department in collaboration with administration.

32. I am new hire ESL teacher as of 11/96. I have not been involved in the decision process as I hope to be in the coming 97/98 school year.

33. I should do some one who have experienced/understand the difficulty of being a second language teacher and or similar background like the students (little or no formal education, background, uneducated parent, and little resources at home).

34. We looked at their time in the US and if they were on an IEP.

35. 2 meeting times.

36. I don't know what the process was that determined who would participate and who wouldn't.

37. I don't know.

38. I am not aware of any process in my building.

39. I don't know.

40. I don't know what the process was.

41. I don't know. I'm not clear on what the process was.

42. I don't know. I'm not clear on what the process was.

43. It was an individualized process, but since most will need to pass the test before they graduate, it was decided that they all should take the test.

44. Administration notified ESL Counselor before letters were sent out to students, and inquired as to whether translations were needed.

45. I made all take it. It's good practice in test taking and giving them realistic expectations on what they have to learn yet.

46. None of the three students were required to take the test. The one who did had been here less than 2 full school years and he chose to take it. Actually, I'm not sure if we should have let him choose to take it, even though he did quite well.

47. We made decisions by discussion.

48. We called the LEP lead at graduation standards at the Dept. of CFL to ask questions. (re: participation).

49. (Decisions were made at) ESL staff meeting.

50. Administration could notify dates of testing more in advance. It takes time to communicate with parents and students.

51. Write out some bottom line criteria.

52. Beginning the process earlier, discussing the test with the students and parents beginning in 6th grade.

53. Change testing date to spring to allow for adequate assessment of student needs. We did not have a district committee (LEP teachers, parents, etc.) in place to make participation decisions this year.

54. It seemed like it was a last minute add on to the testing procedures. It needs to be defined and in place early.

55. Have students take (test) more often.

56. Earlier decisions.

57. A meeting with the LEP district staff giving guidelines and information about the test and who should be tested would be helpful.

58. The (decision making) process was adequate only if the results will be used to show the students' progress from year to year and not to keep them from graduation.

59. The process is adequate as is.

60. (The process is) Adequate as is.

3. Comments on Accommodations Recommendations

1. the use of bilingual dictionaries is highly desire by the students and not over used.

2. Translations with a bilingual dictionary (are recommended).

3. Translations via a bilingual dictionary (are recommended).

4. Bilingual dictionaries on the math test would be very helpful.

5. The use of bilingual dictionaries.

6. Dictionary.

7. Dictionary.

8. They need a bilingual dictionary.

9. Visuals may be helpful with some students.

10. Visual aids.

11. The use of visuals in test instructions...would be very helpful.

12. The use of visuals.

13. Pictures/Illustrations that go with the reading selections. Establish the context of the reading selection.

14. Visuals.

15. Visuals.

16. More Audio-Visual aids.

17. Paraphrase directions.

18. Paraphrasing the questions.

19. Reword the directions.

20. Paraphrase directions so they really understand.

21. Repetition of the instructions in several forms.

22. Read the test to the student, clarifying or rewording as you go (the clarifying and rewording would not have to if this is not appropriate).

23. More examples.

24. Provide more samples (for practice) for the reading test.

25. Giving examples; using culturally familiar examples.

26. Provide more models and samples to prepare them.

27. Each sections of the test, according to my opinion, should contain at least 3 examples for clarity purposes.

28. Have the administrator demonstrate a few examples of each topic testing (sic) before proceeding with the test.

29. Use an OHP to model an example.

30. Explaining the answer in the example before continuing. Being sure the directions and the samples are understood.

31. The need more time.

32. Time is very important, most LEP students need more time.

33. Allow more time.

34. They need more time.

35. They need more time.

36. Small groups or one-to-one (testing) best.

37. It is my belief that the most helpful thing we did for my LEP students during testing was allowing them to test with other LEP students. They did not feel rushed as they would have taking the test with mainstream students.

38. Provide small comfortable group setting (comfortable as in familiar).

39. Include a section that is at a lower reading level than 8th grade to allow students at a lower reading level to have some success.

40. Keep instructions in simplified English posted.

41. Seat LEP students close to test administrator so they can see/hear instructions clearly.

42. Computer assisted learning is appearing to have remarkable effects upon emotionally or learning disabled individuals.

43. Translators (ed. assistants) are used in the classroom, however, I don't recommend this for testing. It sometimes becomes a crutch and given the test, I don't think this is appropriate.

44. We need to provide interpreters.

45. Provide translation in math test; allow staff with students' language to give the direction.

46. The instructions or questions to the problems should be explained in the students' native language so that the results of the test give meaningful assessment of the student's performance.

47. I believe the best for all is to give the instructions in their native language. Tape recording be used or person may be used, which is much better than the tape recording.

48. What about letting the translator see the test a day early to look up awkward vocabulary?

49. Use translation when needed in the content area of studies.

50. The examination has to to be translated, not by words, but having somebody who can explain the meaning of the difficult words, because a student may not grasp the concept.

51. Instructions should be given in a language students can understand. Many failed due to the language barrier.

52. To build the reading and writing of the student They have to have a translator (on the test) because mostly they can not understand the language. So they can not grasp the concept of that mathematics.

53. I recommend they should provide translators for a new American student who don't read and xxx (illegible). Also, one-to-one testing.

54. The time was not right. Tests should be at the end of the year.

55. A simplified ESL test would enable them to (practice before they are proficient enough to really take the test). If they are exempted for 3 years and then suddenly have to take it (as an 11th grader, say) they might not do as well as if they have practice.

56. Specific sill levels would be helpful (such as CAT test reading %).

57. The reading and writing test is too high for LEP students.

4. Comments on Accommodations Decisions in 1997

1. The current belief exists. Work on the test, and use accommodations for testing at the next level.

2. While we didn't make any accommodations (as he [the LEP student] didn't even have to take it) I now feel that we could have used the English tape recording of the math test and we could have given him more time on both tests. In the future, I will recommend both accommodations even for students who aren't required to take the tests. According to the proctor, the child did not appear to need more time and completed the tests in the allotted time.

3. We had all of our LEP students take the test without accommodations.

4. The ESL students were treated the same as regular students once the passed the 3 year exemption period. It's unfair. We have about 40 different languages spoken here besides English. I know that it (the test) was done without any translation.

5. No accommodations were made in the reading test.

6. No accommodations were considered. We were unaware of the options.

7. There were no accommodations made.

8. I'm not aware of any accommodations being made for LEP students.

9. It was determined that no special accommodations were needed.

10. None made.

11. None were made this year. By the time we were informed of guidelines there was not sufficient time to do an in depth analysis.

12. We had the student try without accommodations.

13. Determined that all students were 3 plus years in English instruction/all were tested without accommodations or modifications.

14. None used.

15. No accommodations were made because he didn't have to take it. He chose to, so it was looked upon as an experience to help prepare for the real attempt later. The student also felt that way.

16. The LEP students were mentioned at a Special Ed accommodation meeting which I was unable to attend. As an ESL teacher, I wished I had more information about which was appropriate.

17. We should be better informed of the options.

18. Did not know audio cassette of math was available.

19. We had all of our LEP students take the test without accommodations.

20. I'm not sure if any accommodations or considerations were offered or used.

21. My students were scattered throughout many rooms, so I don't know about them (during testing).

22. Students were tested with mainstream classes. I am not aware of any of the above accommodations being made except for extended time for a few (not necessarily ESL students).

23. I'm not sure if there were any considerations or accommodations.

24. I can't say with even any remote hint of accuracy; only that it was or wasn't used.

25. I don't know when these (accommodation) decisions and guidelines were decided. I know teachers were not included in this. As you and I know, there is little or no LEP teacher involvement in this decision.

26. I am/was not privy to any information/results - I can't answer.

27. I don't know.

28. I don't know.

28. I don't know. I have not been involved in such decision making and I'm not aware of such decisions being made in the building.

29. I don't know.

30. Don't know.

31. I don't know.

32. I don't know.

33. I am not sure - there were many different discussions.

34. I don't know.

35. I don't know.

36. I don't know.

37. I don't know.

38. I have not been told.

39. Don't know.

40. I am not sure. Don't know.

41. Don't know.

42. I don't know.

43. I have no idea.

44. Mainstream counselors administered tests to all students. This included LEP because non opted for the accommodations which were explained and offered by the ESL counselor in advance of test date.

45. I offered accommodations but none of the students wanted to use them. Also, a few students who had not yet been here for 3 years asked to take the tests.

46. The students did not opt for any of the possible accommodations.

47. Instructions from the state LEP lead on Grad Standards were disregarded and overridden by the math teacher responsible for administering the test. The principal overrode my decisions about which LEP students should take the test based on her judgement of the students' performance possibly harming the overall school statistic for 8th grade performance.

48. The morning of the test the principal reversed my decision and disqualified all LEP students not in the county for 3 years from taking the test. This was in lieu of harmful results to school statistics and public relations. Decisions were made based on benefit to the school and teachers, not what was best for LEP students.

49. The people who decide the accommodations for LEP students are not people who truly understand language learning and all the variances from culture to culture.

50. Because of the way the (accommodation) decision went, I assume that the decision makers were not aware of LEP student needs and limitations.

51. I was informed about the (accommodation) options, but I was not part of the decision making group.

52. Make sure you talk to parents so they understand the importance of these tests. They help focus the students (don't just send note).

53. (Accommodation decisions were made by) articulations with the school staff on the basis of student evaluations.

54. (Accommodation decision were made by) ESL teachers and testing team.

55. (Decisions were made by) ESL teachers and testing team.

56. (Decisions were made by) LEP teams of school.

57. (Decisions were made by) Counselors with ESL teachers.

58. Departmental decision (accommodations).

59. (Decisions were made by) ESL teachers.

60. (Decisions were made by) students, staff and parents.

61. District decision, teacher involved in process (accommodations).

62. District provided guidelines and inservice (re: accommodations). Also surveyed parents and community.

63. Through district guidelines. The district decided (accommodations).

64. Downtown administration decides (accommodations).

65. (Decisions were made) as an ESL team.

66. (Decisions were made by) ESL teacher discussion.

67. (Decisions were made by) conferencing with staff members and parents.

68. We had a meeting involving principals, counselors, ESL, district personnel, teachers and a parent representative to discuss and decide upon these (accommodation) issues.

69. ESL teachers looked at several criteria and the ESL coordinator and school administrators made final (accommodations) decisions.

70. (Decisions were made by) conferencing - parents and staff.

71. (Decisions were made by) meeting with ESL, counseling and administrations staff. Final decisions were made by administration (math).

72. (Decisions were made by) group discussions.

73. (Decisions were made by) discussion between those involved in making the decisions.

74. Teachers who work with the students need to make their decisions based on physical, language, and cultural differences of individuals. Accommodations made for the total group will undermine test results of the majority taking the test in that particular group.

75. Decisions (about accommodations) were made across the board for all LEP test takers.

76. We discussed the (accommodation) issue as a department.

77. Accommodations form used: teachers, counselors (a district process) with input from the staff.

78. I looked at the 8th graders needs - most were upper level students and needed only - or could have used a dictionary.

79. ESL/bilingual teachers surveyed LEP students - gave them sample tests with accommodations to see if students thought they would be helpful.

80. LEP students were tested in a room with an available bilingual person to help explain test.

81. Counselor discussed (accommodations) with student - state options - none were chosen.

82. Counselor talked to individual students about their particular needs (for the test).

83. Some students had extended time for reading test - not sure about writing or math.

84. Respondent indicated that while there was a Hmong script of test directions there was no privilege for the 42 other languages in the school.

85. Respondent indicated that Hmong was the only language used under category "script of test directions was clarified or translated."

86. All students receiving ESL services and who have been in the US for longer than one year had short segment tests.

87. They took the test in my room. They feel most comfortable with me and could take longer in my room.

88. (re: translation) Not all cultures - only 1 Hmong, there are 43 language groups here.

89. One problem on the test, when I translated the math questions to a Hispanic student: I occasionally came across words that I did not know and could not translate very well like "shares" when talking about the stock market. It put the student at a disadvantage to not have a perfect explanation, and yet I could not access the test beforehand in order to be prepared. This could happen to both a native and non-native speaker.

5. Comments on Test Preparation

1. Use tutors to work with students on a one to one basis (or very small groups), a thematic approach utilizing culturally-based examples.

2. Use tutors, a thematic approach.

3. Use a thematic approach.

4. Provide meaningful year round/or summer school to the LEP kids. Small class size. ESL/bilingual teachers and translations for those who are willing to get help after school.

5. (Have a) program to help the students specifically for taking the test (after school).

6. LEP students should (be) educated in the methodology to take the test itself.

7. Story, basic Math. These are the basic courses required by the district. I think there should be a special class for this particular test.

8. Test prep classes.

9. Provide classes to the students to teach test-taking strategies.

10. Special classes on preparation, skills.

11. LEP students need additional preparation time and tutoring sessions to familiarize themselves with test formats and procedures.

12. In my intermediate and advanced classes we carefully read the test preparations booklet. We reviewed test taking strategies and discussed answers to the questions. We practiced skimming and scanning skills. I modeled my approach to thinking through a questions requiring inference. We practiced modeling and discussing non-fiction articles. We practiced separating fact from opinion (difficult for all adolescents). We discussed the author's perspective in non-fiction and the character's point of view in fiction. We tried to draw some conclusions from the reading material.

13. Pre-reading activities to build background schema (vocabulary, context, cultural nuances).

14. Rereading, skimming, previewing, underlining.

15. Reading slowly; practicing test taking before this. Exposing them to the newspaper; disadvantaged children come from homes which do not have a newspaper. Teaching test taking strategies!

16. Pre-reading, critical and personal reflection on topics, and oral sharing of ideas develop comprehension and involvement with a reading passage for all readers, not just LEP. Direct vocabulary, cultural and academic elements need teaching before you can expect comprehension.

17. Teach reading strategies.

18. Reading newspaper articles and identifying main points and summarizing also seems beneficial.

19. More math homework.

20. Work on word problems and vocabulary in math seems helpful.

21. In our high school, we have written a translation of math terms into easily understandable English and give it to students before they take the test.

22. Offer awards to the students who do their work consistently.

23. (Teach) self-concept, time use.

24. Do more writing assignments - short and simple.

25. Needs vary according to the students' origins. Some need basic vocab building more. Others need more indirect instruction in reading strategies. Others struggle with sentence structure.

26. A test is a test. Strategies used during typical instruction are tools for learning. Tests measure what has occurred. Basic Standards Tests are not tools for learning; they are measurements of that learning. Don't confuse the two.

27. Not familiar with the strategies that ESL/teachers use that are different for LEP students.

28. LEP students use same strategies as other students. KWL reading strategies, practice for analysis.

29. More practice materials that are more realistic.

30. Provide practice materials before hand.

31. Test prep materials.

32. Practice tests for each part.

33. Providing enough samples tests, and tutoring, and study time.

34. Students should be given a prepared sample test ahead of the actual test.

35. A sample test set up in the identical manner 2 weeks to 10 days previous to the test.

36. Review past tests to get to understand the form and type of questions they need to be ready for.

37. We do the practice testing, using a test best test also, which is a practice test to teach strategies on multiple choice-type testing. We talk about good strategies to use when taking tests of this type.

38. Parallel tests, teaching test taking skills, teaching frequently used test words, (e.g., author, refers to, all except) practice with students so students know/can expect test format; explain process of testing, multiple choice, types of questions.

39. Teach test taking skills: deductive reasoning, finish what you know first, be sure you understand the directions before you start.

40. Students are given the sample test to practice. Strategies are practiced. ex: read questions/activities first for key words; ID author's purpose; practice drawing conclusions, summarizing. Use overhead for class discussion with samples.

41. Teach test-taking strategies.

42. Students need to be familiarized with the format and kinds of directions which are given on standardized tests.

43. It might be a good idea to preview the test the day before (with the ESL students) so the ESL students can see its format. Then if they are confused about it in any way, the issue can be addressed in a small group where they won't be hesitant to ask or be embarrassed.

44. Practice items on the test.

45. LEP students should be taught study skills and test taking skills in the native language or in the second language. These skills are so essential to their educational success.

46. They need to review the questions first before testing.

47. Student has to be given opportunity to do sample tests before they take the actual tests. There should be days designated to practicing this test for teachers/students.

6. Comments on Needs

1. Need for common data base and common identification procedures.

2. Have accurate data on date of arrival in U.S. Some confusion when students have moved from another city or state.

3. Accurate data - US previous education and English instruction; accurate definition of LEP student or students receiving ESL services.

4. I don't believe LEP students should take the test.

5. Need more data prior to test.

6. Teachers need reading levels topic areas and questions - type information to decide if their students will benefit form this assessment. Why give a 1st year reader a middle school reading test?

7. At May 96 state LEP conference we were told we would be contacted before 9/96... yet none of the LEP teachers at our building were contacted.

7. Miscellaneous Comments

1. Different cultures have different needs. A 12 year old from France would be better educated than a 12 year old from Laos and could do better on the Basic Standards Test.

2. At our school we have 6 levels of ESL (0-5). Not one student in level 4,3,2,1, or 0 passed (some in Level 4 were close). It isn't lack of trying, intelligence or preparation. They haven't had the time (5-7 years) to achieve academic competence in English.

3. Students should be able to pass the test as well as any other. If not they should get extra help.

4. Too many students were confused, misinformed, frustrated, distressed and embarrassed by the whole process.

5. One of our LEP students is being referred for more ESL. Had been exited but basic tests indicated further needs of services.

6. The pressure of approximately 15 testing dates in school year, variable testing requirements, inappropriate testing materials, and inadequate preparation for them - all make your survey appear ludicrous.

7. I was expected to identify which LEP students should take the test. This is very time consuming, it take away from teaching time.

8. Reading comprehension is very difficult to LEP students.

9. It is my understanding that ESL students can request NO special help once they enter the test taking requirements, which is being in America 3 years or longer.

10. We are most concerned about ESL students facing the writing test.

11. I don't like that LEP scores are absorbed into the general number when low scores are reported in the media. What would you expect when students with little English are tested?

12. Surveying our opinions 4 months after the fact assures invalid results.

Top of page