Participation and Performance of Students from non-English Language Backgrounds: Minnesota's 1996 Basic Standards Tests in Reading and MathMinnesota Report 17Published by the National Center on Educational OutcomesPrepared by Kristi Liu, Martha Thurlow, Sandy Thompson, and Deb Albus January 1999Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Liu, K., Thurlow, M., Thompson, S., & Albus, D. (1999). Participation and performance of students from non-English language backgrounds: Minnesota's 1996 Basic Standards Tests in reading and math (Minnesota Report No. 17). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/MnReport17.html OverviewThe majority of states in the United States have some type of statewide assessment. Minnesota has had the Basic Standards Tests (BST) since 1996. The Basic Standards Tests are tests of basic math and reading skills that a student is required to pass in order to graduate from high school. A student must take these tests for the first time in 8th grade and must pass the reading and math tests by 12th grade in order to receive a MN state diploma. A required writing test administered to students in the 10th grade will be implemented in the winter of 1999. In the assessment literature, a general recommendation for large-scale assessments has been to disaggregate scores and other data for students who have limited English proficiency (LEP). Minnesota does not consistently gather and report achievement data on LEP students as a total group. Furthermore, given the recent changes in immigration trends for the midwestern area, educators are also starting to call for data showing how various sub-populations of the total group are performing. Nationwide, Spanish speakers make up the largest proportion of students with home languages other than English (some of whom are identified as LEP and some of whom are not). Therefore, the majority of research studies that have examined sub-populations have looked at the achievement of Spanish-speaking students on large-scale assessments. In contrast to the national statistics, Southeast Asian students make up the largest percentage of students with a home language other than English in Minnesota. Clearly there is a need for information about the large-scale assessment performance of students from other language groups as well. The National Research Council (1997) supports this need for research that examines specific language groups in addition to Spanish speakers. It is important to move away from a “one size fits all” approach (Lucas, 1997) when examining the participation and performance of non-English language background (NELB)* students in statewide assessments and using their achievement data to make educational programming decisions. Educators and policymakers need to recognize that even though groups of NELB students may share some common characteristics, in many ways their needs are still very diverse depending on their ethnic backgrounds, their educational histories, their migration status, etc. These diverse and complex needs cannot be accurately captured in a single statistic showing the performance of NELB, and in particular, LEP students, overall on a statewide assessment. This document reports on the achievement of NELB students from the largest seven language groups in the state of Minnesota during the 1995-96 school year when Basic Standards Tests were first implemented. The data are also broken down into whether students received English as a Second Language (ESL) or bilingual services during that year. While the achievement data reported here may not be surprising to the ESL and bilingual teachers who work most closely with students from the seven language groups, these educators are often not involved in making decisions about the statewide assessment program (Liu, Spicuzza, Erickson, Thurlow, & Ruhland, 1997). It is the hope of the Minnesota Assessment Project that, with this report, educators and policymakers can begin to examine the differing performance levels and needs of NELB and LEP students instead of assuming that the needs of all these students are the same. Method The data used in this report were provided by the Minnesota Department of Children, Families and Learning (CFL). Only the data for eighth grade students who took the test during the 1996 Basic Standards Tests and who were in the database at the CFL were available for analysis by the Minnesota Assessment Project. In this document the terms Non-English Language Background (NELB) student, Limited English Proficient (LEP) student, and English as a Second Language (ESL) services are defined in specific ways, which may differ from the ways in which others use the terms.
Non-English Language Background Students An NELB student is a student who speaks or has had significant exposure at home to a language other than English. Defining students as “NELB” does not necessarily mean they have limited English proficiency, nor does it differentiate between students who may need ESL or bilingual services now or who may have received services in the past. Elsewhere, there is no consistent use of a term for students whose home language is a language other than English. Some use the term “Linguistically Diverse” students (Hamayan & Damico, 1991). Others have used “Persons Whose Home Language is Other Than English,” (PHLOTE) or other categories. Past reports from NCEO have used the term “Language Minority” to refer to these students because it is the term used by the federal government. However, in this document and in future documents, the term NELB will be used instead of Language Minority.
Non-English Language Background Students An LEP student is an NELB student who performs below the level of his or her native English speaking peers when assessed on a nationally normed standardized test in English. This low level of English ability makes it difficult for the student to make gains in English-only instruction. The term “LEP,” although controversial because of its concentration on a student's lack of ability in English, is the term chosen in this report because both the federal and state of Minnesota governments use it (Liu, Thurlow, Erickson & Spicuzza, 1997). Minnesota has the following definition of LEP students in its law:
However, Minnesota school districts may vary in their interpretation of this definition, and the definition is not standard for other states and districts across the country.
Services for LEP Students There are no commonly accepted criteria for defining services provided to LEP students. These services may vary in amount of time for instruction, the type of services offered (e.g., ESL or Bilingual), the method of instruction, and the administration of services. Some students, due to issues of mobility (as in the case of migrant students), also may have had interrupted services or services provided in another school district, whether in state or out of state. Therefore, students who are categorized as receiving ESL or bilingual services in this report may have received a variety of possible services. A range of services in ESL and bilingual programs is outlined below by Cuevas (1996) according to where the services fit in a continuum of instruction, whether English only or non-English languages.
District Criteria for Receiving Services To determine eligibility for ESL or bilingual services, the state of Minnesota uses the legal definition given on pages 3 and 4 of this report. Students who are identified as LEP are eligible to receive services, although not all of them choose to do so. It should be noted that test instruments used to assess English language proficiency and average scores may vary significantly by district. This means that students who are borderline LEP may be defined as LEP in one district, but not in another. This may also occur for students moving within or between states.
Participation The total number of NELB students who took the Basic Standards Test (BST) and who appear in the database is 1,887. However, this is not the total number of NELB students in the state. Participation in the 1996 Basic Standards tests was voluntary for schools in 1996. Therefore, the results that were available for analysis were from those school districts that volunteered to participate in the Basic Standards Tests for that year. A total of 330 of 362 districts tested in math and/or reading in the Spring of 1996. Results should be interpreted accordingly. The data were disaggregated into language groups to examine possible differences between the groups, both in terms of ESL/bilingual services provided and student performance. Because there are few factors that LEP students have in common besides having limited English proficiency, a more insightful approach is to consider language groups separately, due to their varied backgrounds and needs. LaCelle-Peterson and Rivera (1994) state that although LEP students share “the need to increase their proficiency in English—they differ in language, cultural background and family history” (p. 59). Because of these differences, each language group “may interact with schools differently.” There are approximately 67 language groups in the database kept by the CFL. Of this total number, the data for the seven largest NELB groups, based on the student population for grades K-12, were run on SPSS, a statistical data analysis computer program, to determine the percentage of students in each group who scored at or above 70% on either math or reading sections of the BST and whether they were receiving services. Those language groups are Hmong, Spanish, Vietnamese, Lao, Cambodian, Russian, and various African languages. It should be noted that in this testing year African languages were not separated by the various languages represented in that category. In April of 1996, this category was changed to make distinctions between language groups such as Somali, Swahili, Amharic, Tigrinian, Ibo, and Yoruba. Therefore, the African language group in this report should be understood to include all eighth grade African language students who were categorized as NELB students in 1996. Table 1 shows the total number of eighth grade students in the seven largest NELB groups, as provided by MINCRIS, a student demographics program that includes language background. The number and percent of students taking the math and reading tests are based on the total number of students who could have taken the BST reading and math tests in 1996. Table 1 shows that for each NELB group, except Spanish, at least half of the total number of eighth graders participated in the BSTs in reading and math in 1996. In the Spanish group, the number of students who took the math and reading tests was a little below half of the total number. It should be noted that in the data there are varying percentages of students in each group who took the BSTs. For example, for the reading test, 33 of the 66 Russian students participated (50%), in contrast to 717 of 917 (78%) in the Hmong group. Statistics were run based on the total number of NELB students who took the 1996 tests in reading and math in order to determine the numbers of students who scored at or above 70 percent, and whether they were receiving services. Next, statistics were run on SPSS for specific language groups for scores at or above 70%, and whether the students were receiving services. For that year, both sections of the BST required a passing score of at least 70%. Although the scores are reported based on the 70% criteria, this “passing” rate may not have actually been used by individual districts, which had the option of raising the required passing score. Also, students receiving special education services may have passed at a modified level with an individually determined passing score.
Table 1. Number of 8th Grade Students in Seven Largest NELP Groups
* The total number includes students in all school districts, including those that did not participate in the voluntary testing. Thus, the percentages of students in this table who were tested are not fully representative. Findings Reading Test Table 2 shows the total number of NELB students who took the 1996 BST in Reading and the number and percentage of students, either receiving or not receiving services, who scored at or above the 70% passing score.
Table 2. 1996 Performance of NELB Students on BST-Reading
Even though a higher percentage of students not receiving services had passing scores, it should not be assumed that these students do not need services. There are several reasons for this finding. One reason is that students “without services” may have received ESL/bilingual services in the past, but no longer receive them due to increased English proficiency. For example, a student receiving services in earlier grades could have been exited from a program based on a school's individual exiting criteria. Other students in the “without services” group may be in various stages of exiting a program. A student who is being monitored in mainstream classes for possible exit from a program may not officially be receiving services. It could also be that students may have been deemed “Fully English Proficient” by parents or school evaluators, even though the predominant language at home was a language other than English. This may include students who come from partially bilingual homes. It is also possible that students with limited English proficiency who may have qualified for receiving ESL/bilingual services were not receiving them because their parents chose not to give consent for receiving services. This category includes students whose parents may have denied evaluative testing of their child for possible services. As seen in Table 2, fewer of the students receiving services are scoring at the 70% passing level than are students not receiving services. This may be a result of lower reading comprehension skills as well as low English language skills such as unfamiliarity with American English idioms and vocabulary. Lower scores could be expected from students who are working to increase general English proficiency at the same time that they are learning how to read for main ideas. It is also possible that the quality of ESL/bilingual programming that a student receives may be another factor that is interacting with factors such as high mobility rates, lack of appropriate reading or math instruction, or living in an environment of poverty, all of which also may result in lower test scores. The numbers in Table 2 can be examined in another way, as shown in Table 3. Viewed this way the percentage of students passing the Basic Standards Test in reading was made up of 12% students receiving services and 88% students not receiving services. Those not passing the test consisted of 51% students receiving services and 49% students not receiving services. It is possible that students of that 49% could have benefited from services by gaining the skills needed to pass a basic reading exam.
Table 3. Placement in Services of Students Passing and Not Passing the 1996 BST in Reading
Table 4 shows each language group's results on the BST reading test, both overall and as a function of whether they were enrolled in ESL/bilingual services. In general, the data show it was difficult for students from all language groups to score at or above 70% for reading. Consistent with previous tables, students not receiving services scored at or above 70% more often than students who were receiving services. By groups, the table shows that Russian speaking students, overall, had the highest percentage (42%) of “passing” scores, followed by the Vietnamese and Cambodian students at 37% and 36%, respectively. These differences by language group may reflect a variation in general language background and literacy in a student's first language. For example, students from Russia often enter American schools with more educational background in their native language than do students from other countries. The group that had the lowest percentage scoring at or above 70% was the African languages group, with only 15% of the students scoring at least 70%. However, this group also had the second smallest total number who took the test, with only 47 students. In addition, it's important to keep in mind that many of the students in this group are the most recently settled refugees in the Midwest and may still be in the process of adjusting to life in a new country. The Spanish-speaking group had the next lowest percentage of “passing” students at 24% out of a total of 369 students who took the test.
Table 4. Percentages of NELB Students Receiving or Not Receiving Services with Reading Scores At or Above 70%
For students who were receiving services, the Russian and Spanish speakers scored above 70% more than other language groups (27% and 13% respectively). This shows that even though Spanish speakers scored the second lowest in reading, those that received services had the second highest percentage of students scoring at or above 70% in reading.
Math Test Table 5 presents the number of NELB students, with and without services, who took the 1996 BST in Math and the number and percentage of those who scored at or above 70%. As in Table 2, students not receiving services scored at or above 70% at a higher rate than did those students receiving services. As for reading, there are many possible reasons for this finding.
Table 5. 1996 Performance of NELB Students on BST - Math
Not only did a larger number of students participate in the math test, but students tended to score better on the math test, overall, in both categories of services. A comparison of Tables 2 and 5 highlights the differences between the results of the two tests. In percentages, approximately 34% scored at or above 70% for the reading test in contrast to the 47% who had a passing score for the math test; 13 percent more students received passing scores for math than for reading, which would suggest that the reading test is more difficult to pass. Also, there were 33 more students who took the math test than the reading test. This variation in the total numbers of test takers for each test could be due to several reasons. Because the two subtests were given on different days, students may have been absent on one of the days. Also, it is possible that schools encouraged more of their students to take the math test in the belief that students would perform better on the math test than on the reading test. The number of students who took both subtests is not available from the data that we received; however, because 1996 was an optional testing year, the two populations in the results presented here for reading and math scores could have been different. The numbers in Table 5 can be examined in another way, as shown in Table 6. Viewed this way, the percentage of students passing the Basic Standards Test in Math is presented with the percentage of students not passing, categorized by receiving and not receiving services.
Table 6. Placement in Services of Students Passing and Not Passing the 1996 BST in Math
The numbers in Table 6 do not necessarily raise the questions that the numbers in Table 3 did. For example, the fact that 21% of the students passing the math test were receiving services does not necessarily indicate that they do not need services. For example, a student who passes with basic skills in math may still need language support. On the other hand, the 46% of students not passing and not receiving services does lead to questions about why those students are not receiving services. If poor scores are related to language access issues, the need for services is supported. Table 7 shows each language group's results on the BST in math, both overall and as a function of whether they were enrolled in ESL/bilingual services. Each language group performed better on the math test than on the reading test, which indicates that the math test is less difficult for NELB students to pass. As in reading, the percentage of “passing scores” was higher among the Russian group (79%) and the Vietnamese group (55%). Although, compared to the reading test, there was a general increase in “passing” scores for the math test across all language groups, the Spanish and Hmong groups had the lowest increases at only 8% and 9% respectively.
Table 7. Percentages of NELB Students Receiving or Not Receiving Services with Math Scores At or Above 70%
Discussion Effectiveness of Services The data presented here cannot be used for forming judgments about the effectiveness of ESL/bilingual programs due to the complex nature of services provided. For example, although the CFL database contains descriptions about the types of ESL/bilingual services offered, the programs cannot be equated with the descriptions given earlier in this report (pages 4 and 5) because programs at schools may offer combinations of models, or may vary in how strictly they follow an adopted model. There may also be little descriptive information about how many hours or minutes per week a student receives services. In addition, the regular school year programs may be supplemented by other services such as extra-curricular programs during the school year or summer programs. These programs, likewise, may vary in the types of services provided.
Minimal Requirements It should be understood that the requirement met by passing the Basic Standards Tests is the minimal requirement for graduation. Of the total number of students who took the tests, only 34% “passed” reading and 47% “passed” math. This means that not even half of the NELB students who were tested, with or without services, were able to score at or above 70% on the reading or math tests. Further, students will also need to pass a Basic Standards Test in writing beginning in 1999. It is possible that students who pass the BSTs in math and reading may need more help in writing because this is a generative skill that may be more difficult to acquire. Another challenge for graduation within the next few years, in addition to the BST, is the successful completion of 24 High Standards, a performance based system implemented in the classroom and community. It is possible that students scoring at or above 70 percent on the BST in math and reading may have difficulty with the High Standards, which requires students to achieve beyond the minimal requirements of the BSTs.
Placement Issues As mentioned previously, just because a student is not currently receiving services does not mean that the student is not in need of services in order to achieve a passing score on the Basic Standards Tests. Currently there is not much information available about issues involving Minnesota students' placement in and exit from ESL or bilingual services. In a study of California public schools, Gandara and Merino (1993) found that there was conflict between school or district policies on the reclassification of LEP students as “Fully English Proficient” (FEP) and what happened in actual practice. Because of practical issues such as a lack of funding and the resulting inability to test students as often as needed for reclassification, as well as a correlation between funding and the number of students exited from LEP programs, schools often did not follow the reclassification policies. This resulted in students who were kept in ESL or bilingual classes until teachers thought that they were “ready” to exit, and then students were exited without being tested. The researchers also found that some students were permanently kept in ESL programs because teachers believed that students were best served there in spite of their advanced English proficiency. Other students who did not necessarily meet exit criteria were exited because of their higher proficiency levels when the school or district had large numbers of students with comparatively lower proficiency levels. California does have a higher overall percentage of LEP students than Minnesota does, which might reduce the possibility of these issues occurring in Minnesota, but it is possible that Minnesota schools and teachers are facing the same conflicts of theory and practice as they attempt to serve a rapidly growing LEP population. If, in fact, these types of reclassification difficulties are happening in Minnesota, looking at BST performance broken down by students who are and are not receiving ESL or bilingual services may not be giving a complete picture of the students' needs. Students who are getting services may or may not continue to need them, while students who are not getting services may still be in need of them. This is especially true since the Basic Standards reading test only measures a student's ability to read for information. Other parts of Minnesota's graduation standards require additional types of reading skills that the BST does not measure; students who pass the BST may not possess these other types of reading skills.
Instructional Implications There is not much literature available that gives specific instructional implications of the issues raised in this report. The literature that does exist tends to be program-wide. A few resources with program-wide recommendations for improving literacy skills are listed below: Grossen, B. (undated). Thirty years of research: What we now know about how children learn to read. A synthesis of research on reading from the National Institute of Child Health and Human Development commissioned by the Center for the Future of Teaching and Learning with funding support from the Pacific Bell Foundation. [On-line] Available: http://www.cftl.org/30years/30years.html. Molina, H., Siegel, D. F., & Hanson, R. A. (undated) Early reading instruction, non-English language background, and schooling achievement. [On-line] Available: http://www.ncbe.gwu.edu/ncbepubs/symposia/third/molina.html. National Research Council. (1998). Preventing Reading Difficulties in Young Children. Washington, DC: National Academy Press. Roberts, C. A. (1994). Transferring literacy skills from L1 to L2: From theory to practice. The Journal of Educational Issues of Language minority students, 13, 209-221. [On-line] Available: http://www.ncbe.gwu.edu/miscpubs/jeilms/vol13/transf13.html. The following resource addresses ways to improve education for students at-risk of school failure: Tharp, R. G. (undated). From at-risk to excellence: Research, theory, and principles for practice. [On-line] Available: http://www.cal.org/crede/pubs/rrl/rrl.htm. References Cuevas, J. (1996, January). Educating limited English proficient students. (PD-96-01). San Francisco, CA: WestEd. Gandara, P., & Merino, B. (1993). Measuring the outcome of LEP programs: Test scores, exit rates, and other mythological data. Educational Evaluation and Policy Analysis, 15 (3), 320-338. Grossen, B., (undated). Thirty years of research: What we now know about how children learn to read. A synthesis of research on reading from the national institute of Child Health and Human Development commissioned by the Center for the Future of Teaching and Learning with funding support from the Pacific Bell Foundation. [On-line] Available: http://www.cftl.org/30years/30years.html. Hamayan, E. V. & Damico, J. S. (Eds.) (1991). Limiting bias in the assessment of bilingual students. Austin, TX: Pro-Ed. LaCelle-Peterson, M. & Rivera, C. (1994). Is it real for all kids? A framework for equitable assessment policies for English language learners. Harvard Educational Review, 64 (1), 55-75. Liu, K., Spicuzza, R., Erickson, R., Thurlow, M., & Ruhland, A. (1997). Educators' Responses to LEP Student Participation in the 1997 Basic Standards Testing (Minnesota Report 15). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Liu, K., Thurlow, M., Erickson, R., & Spicuzza, R. (1997). A review of the literature on students with limited English proficiency and assessment (Minnesota Report 11). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Lucas, T. (1997). Into, through, and beyond secondary school: Critical transitions for immigrant youths. McHenry, IL: Delta Systems, Co., Inc. Minnesota Statutes. (1996). 126.262 [On-line]. Available: http://www.revisor.leg.state.mn.us/st96/126/262.html. Molina, H., Siegel, D. F., and Hanson, R. A. (undated). Early reading instruction, non-English language background, and schooling achievement. [On-line] Available: http://www.ncbe.gwu.edu/ncbepubs/symposia/third/molina.html. National Research Council. (1998). Preventing reading difficulties in young children. Snow, S. E., Burns, M. S., Griffin, P., (Eds.). Washington, DC: National Academy Press. National Research Council. (1997). Improving schooling for language-minority children: A research agenda. August, D. & Hakuta, K. (Eds.). Washington, DC: National Academy Press. Roberts, C. A. (1994). Transferring literacy skills from L1 to L2: From theory to practice. The journal of educational issues of language minority students, 13, 209-221. [On-line] Available: http://www.ncbe.gwu.edu/miscpubs/jeilms/vol13/transf13.html. Tharp, R. G. (undated). From at-risk to excellence: Research, theory, and principles for practice. [On-line] Available: http://www.cal.org/crede/pubs/rrl/rr1.htm. |