Participation and Performance of Limited English Proficient Students: Minnesota Comprehensive Assessments in Reading and Math, 1998-99Minnesota Report 24Published by the National Center on Educational OutcomesPrepared by Kristin Liu and Martha Thurlow January 2000Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Liu, K., & Thurlow, M. (2000). Participation and performance of limited English proficient students: Minnesota Comprehensive Assessments in Reading and Math, 1998-99 (Minnesota Report No. 24). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/MnReport24.html Overview Title I of the Improving America’s Schools Act (formerly
known as the Elementary and Secondary Education Act) mandates that all states develop high
standards for the academic achievement of their students, and that they include all
students, particularly those who are low achieving. Furthermore,
the law specifies that educational accountability will be improved through the use of
state assessment systems that have been designed to show how well students are achieving
challenging State student performance standards expected of all children (United States
Code Service [USCS], 1999). According to the National Research Council (1999), standards
provide a way to focus support for schools that have difficulty educating all students to
the same high level. The National Research
Council calls on states to monitor their own standards implementation and to make changes
where needed. One group of students that has
particular difficulty achieving the high standards is those students who are English
language learners (also called limited English proficient students or LEP students). These students are learning the English language
at the same time they are trying to learn content material needed to achieve the
standards; therefore, focused examinations of LEP students’ participation and
performance in state standards-based assessments are important for determining the types
of support they need to achieve at high levels. The
Minnesota Assessment Project is a four-year, federally-funded award from the Office of
Educational Research and Improvement that has published more than 12 documents relating to
the participation and performance of LEP students in Minnesota’s standards (see
Appendix A for a list). Minnesota has developed a system of high standards that are being
implemented at the high school level. Students
must complete standards in specific areas to be eligible for a high school diploma. In order to determine whether elementary and
middle school students are learning the concepts and skills needed to complete the high
school standards when they are older, the Minnesota Department of Children, Families and
Learning (CFL) has created preparatory standards for kindergarten through grade eight. These preparatory standards include the areas of
math, science, the arts, reading, writing, and speaking.
An example of the standards in the area of reading at grades 3 is listed as
follows: PREPARATORY CONTENT STANDARDS IN LEARNING AREA
ONE: READ, LISTEN, AND VIEW. (Grade 3) Literal
Comprehension A student shall demonstrate comprehension of
literal meaning by: 1. reading,
listening, and viewing of nonfiction and fiction selections to identify main ideas and
support details, retell main events or ideas in sequence, pronounce new words using
phonics, demonstrate techniques of improving and expanding vocabulary, and demonstrate an
age-appropriate reading rate; 2. reading
and applying technical instructions to perform an action; 3. using
presentations of data to understand scientific or mathematical information; and 4. summarizing
ideas and information from visual presentations. Interpretation
and Evaluation A student shall interpret and evaluate
information from age-appropriate nonfiction and fiction selections by reading, listening, and
viewing to: 1. distinguish
fact from opinion in nonfiction selections; 2. interpret
figurative language; 3. make
predictions based on information in the selection; 4. compare
and contrast settings, ideas, or actions; 5. understand
ideas not stated explicitly in the selection; and 6. interpret
effects of persuasive visual messages. (http://cfl.state.mn.us/GRAD/highstds/IL-RVL.htm) To assess progress on the preparatory standards, CFL also has
developed and implemented the Minnesota Comprehensive Assessments (MCAs) of reading and
math in grade 3 and of reading, math, and writing in grade 5. The MCAs are criterion-referenced tests used for
district accountability purposes, and as a tool to make decisions about curriculum and
instruction. This document focuses on the participation and performance of
students with limited English proficiency in the first cycle of MCA testing in 1998. Only the standardized math and reading tests are
discussed here because they are present at both grade levels. The writing test, which is not included here, is a
performance assessment administered in grade 5 that is scored using a different type of
rubric. The writing test results will be
addressed in a future report. The Minnesota Comprehensive Assessments (MCAs) The MCAs were designed by a team of more than 200 Minnesota
educators plus national experts in reading and math.
The reading and math tests have multiple choice and short answer items. There are 10 forms of the test with 49 common
items (58 possible points) and 11 additional items, called “matrix items,” that
vary across test booklets (14 possible points). When
individual student results are reported, their results are based only on the common items
found on every student’s test booklet. When
school or district results are reported, the results include both the common items and the
matrix items. Tests are untimed and scheduled
over two days, with one test given per day. At grade 3, the math test includes questions in the following
areas: shape, space and measurement; number sense; chance and data handling. For example, to assess the number sense standard,
the test may ask students questions about counting, whole numbers, and identifying odd and
even numbers. The reading test covers
informational material (e.g., textbooks), practical information (e.g., recipes), and
literature. On average, test items are
written at a 3rd grade reading level. At grade 5, the math test includes items in the same areas as
those assessed in grade 3: shape, space and measurement; number sense; chance and data
handling. However, the types of problems are
different. For example, to assess the number
sense standard at grade 5, students may be asked questions about place value, number
operations, estimating, and calculator use. The
reading test covers material in the same categories as those on the grade 3 test
(informational material, practical information, and literature), with the average passage
written at a 5th grade level. There is no passing score that all students are expected to
achieve on the MCAs. The tests are designed
to monitor the educational system in Minnesota and each district. Although individual students’ scores can be
reported, the tests are designed for system accountability rather than individual student
or teacher accountability. These state,
district, and school test results can be used in several ways: •
To measure the success of schools and districts in improving
student achievement over time. •
To generate information for school improvement and
accountability. •
To allow for identification of programming and strategies that
keep student performance on track for achieving high standards in high school. •
To allow for comparisons of schools and districts throughout the
state. The results of these tests do not impact students’
graduation or promotion status. Students with limited English proficiency are required to
participate in the MCAs. Only those very few
LEP students who have been in the United States for one year or less may be exempted from
the assessments. These students are
identified through a local school process. LEP
students can take the MCAs as generally administered or with approved accommodations. See Appendix B for a list of these accommodations. Data Reporting There are two types of data available on MCA student performance. First, there is a numerical score. As previously mentioned, each student answers 60
questions for a total of 72 possible points. However,
individual student scores are based only on responses to the 49 common items for a total
of 58 possible points. School or district
results are based on students’ responses to both the 49 common items and all the
possible matrix items, for a total of 159 items and 198 possible points. Numerical scores reported here are scale scores
based on the common items found in all test booklets. Second, data are also available by student achievement level. Title I law specifies that test results must be
disaggregated into at least three categories of performance level: (1) those students
performing at proficiency level, (2) those students performing significantly above
proficiency level, and (3) those students performing below proficiency level. Minnesota has divided students’ MCA scale
scores into four achievement levels: Level
IV - Students at this level demonstrate superior performance, well beyond what is
expected at the grade level. Level
III - Students at this level are working above grade level. Many are proficient with
challenging subject matter. Level
II - Most students in Minnesota fall within this level. This includes a wide range of
students, from those with partial knowledge and skills to students who are increasingly
proficient with grade level material. Level I
- Students at this level have gaps in the knowledge and skills necessary for
satisfactory work. (Minnesota Department of Children, Families
and Learning, 2000) The range of scores in each level varies by grade and test. According to the Minnesota Department of Children,
Families and Learning (CFL), students in levels III and IV have demonstrated the capacity
to successfully achieve the high standards if they continue to perform at this level until
high school. Method CFL collected the data compiled for this report through the
Minnesota Automated Reporting Student System (MARSS).
Minnesota Assessment Project researchers ran descriptive statistical
analyses using the SPSS Information Analysis System. Because of the way MARSS data fields are constructed,
disaggregated MCA data are only available for those LEP students who receive English as a
Second Language (ESL) or Bilingual Education services.
Those students receiving services actually represent a subset of all those
students eligible to receive services. A
small group of students may not receive services even though they are eligible because
their parents or guardians choose not to access these services. In this report, when the reader sees the term
“LEP student” used, it should be understood to mean those students who are LEP and who receive ESL or Bilingual Services. Numbers showing total student enrollment are based on fall
counts. Reports based on other counts may
produce slightly different results. Participation and Performance Third and fifth graders in the 1998-99 school year were the first
group to take the math and reading MCAs. Figure
1 shows that participation rates for LEP 3rd and 5th graders taking both the math and reading MCAs were similar in
1998 and 1999 to the participation rates for all students from grades 3 and 5. Approximately 85% to 90% of students in each group
participated in the MCAs. Table 1 shows
1998-99 MCA participation data in more detail.
The data in Table 1 show that in 1998 and 1999 approximately 4%
to 5% of the enrolled students in grades 3 and 5 were LEP students receiving ESL or
bilingual education services. Roughly 500 to
600 more LEP students were enrolled in grade 3 than in grade 5. Table
1: Participation in the 1998 and 1999 MCA Exams at Grades 3 and 5
Minnesota law requires that all third and fifth-grade public
school students participate in the MCAs, with only a few exceptions. As previously mentioned, students with limited
English proficiency may be exempted if they have been in the United States for less than
one year and students with disabilities participate according to the provisions of their
individualized education programs. Given
these facts, 89% to 96% of both LEP students and all students at grades 3 and 5
participated in both the math and reading MCAs. Based
on the data, it is assumed that the 7% to 11% of LEP students who did not participate
either have been in the country less than one year and were exempted from taking the MCAs
or, because of participation in special education, have an individualized education
program (IEP) that specifies different testing arrangements. Performance Math A range of scale scores determined the four achievement, or
proficiency, levels as follows:
CFL states on its Web site that most students score at Level II
on the math MCAs. This was not necessarily
true for LEP students. In 1998, the majority
of 3rd grade LEP students scored at Level I on the math test,
indicating that they have gaps in the knowledge and skills needed for satisfactory work. Figure 2 shows that in 1998 the majority of
“all students” and in 1999 the majority of both the 3rd grade “all students” group and the 3rd grade LEP students did in fact score at Level II. The percentage of students scoring in other levels
varied considerably between groups, with greater percentages of LEP students in the first
two levels and much smaller percentages in the top two levels. In contrast, the majority of all students
scored at Levels II and III. Table 2 shows a
more detailed breakdown of the proficiency levels of both the LEP students and the
“all student” group on the MCA math test at grade 3.
Table
2. 3rd Grade MCA Math Performance
*All
percentages have been rounded In 1998 and 1999 the majority of the “all students”
group scored at Level II (46% to 47%) and in 1999 the majority of the LEP students scored
at Level II (53%). However, the 3rd grade groups differed in where the second largest portion of the
students typically scored. Twenty nine
percent to 33% of the “all students” group fell in Level III while 38% to 52% of
the LEP students fell in Level I. The third
largest portion of the 3rd grade “all student” group (12% to 18%) fell into Level
I while the third largest portion of the 3rd grade LEP students (6% to 9%) fell into Level III. Finally, the smallest portion of both the
“all student” group (6% to 9%) and the LEP students (1%) fell into Level IV. If students scoring at Levels III and IV in math
are on track to achieve the high standards in high school, only 7% to 10% of the LEP
students are on track to achieve the high standards if they continue to perform at the
same levels. In contrast, 35% to 42% of all 3rd grade students are on track to achieve the high standards if
they continue to perform at the same levels. Figure 3, representing achievement for grade 5, shows variation
between the percentages of LEP 5th graders and all 5th graders who score in each group.
In this case, the largest portion of grade 5 LEP students scored at Level I
(55% to 60%) in both 1998 and 1999. In
contrast, the largest portion of 5th graders in the “all students” group scored at Level II
(45% to 50%) in both years. Table 3 shows 5th grade MCA math achievement in more detail.
Table 3. 5th Grade MCA Math Performance
*All percentages have been rounded In 5th grade, the percentage of students scoring at each level on the
math test differed considerably between the LEP students and the “all students”
group. Most LEP students scored at Level I
(57% to 60%) on the math test, while most of the “all students” group scored at
Level II (45% to 48%). The second largest
group of LEP students scored at Level II (36% to 38%) while the second largest group of
“all students” scored at Level III (26% to 31%).
The third largest portion of the 5th grade LEP students scored at Level III
(4%) while the third largest portion of the “all student” group (18% to 20%)
fell into Level I. Finally, the smallest
portion of both the LEP students (<1%) and the “all student” group (6%)
scored at Level IV. If only those students
in Levels III and IV are on track to achieve the high standards in high school, roughly 5%
of 5th grade LEP students are on track to achieve the high standards. In contrast, 31% to 37% of all students appear to
be on track to achieve high standards in high school if they continue to perform at the
same levels. Reading A range of scale scores determined the four reading achievement,
or proficiency, levels as follows:
Figure 4 indicates that the percentages of students scoring at
each performance level on the Reading MCA at Grade 3 is fairly consistent for LEP students
and all students in 1998 and 1999. The
largest portion of LEP students (60%-65%) score at Level I on the reading portion of the
MCA. In contrast, the largest portion of the
“all students” group (roughly 40%) scores at Level II. The percentages of students falling into other
levels vary considerably between groups. Table
4 shows Grade 3 reading achievement in more detail.
Table 4 indicates that the percentage of 3rd grade LEP students and all students performing at different MCA
reading achievement levels varies widely. The
largest number of LEP students taking the MCA Reading test performed at Level I (61% to
66%) while the largest portion of the all students group (39% to 42%) performed at Level
II. The second largest portion of LEP
students performed at Level II (29% to 31%) while the second largest portion of the total
student group performed at Level III (30% to 32%). The
third largest portion of LEP 3rd graders scored at Level III (4% to 7%) and the third largest
portion of all students scored at Level I (21% to 23%).
Finally, the smallest percentage of both groups (<1% and 6% to 8%,
respectively) scored at Level IV. About 5% to
8% of the 3rd grade LEP students score at Levels III and IV in MCA reading and
would be on track to achieve the high standards in high school if they continue to perform
at a similar achievement level. In contrast,
36% to 40% of the “all students” group appear to be on track to achieve the high
standards. Table 4. 3rd Grade MCA Reading
Performance
*All percentages have been rounded Figure 5 shows the performance of both 5th grade LEP students and of all 5th grade students on the MCA reading test. At grade 5, the LEP students score predominantly in Level 1 (60% to 65%). However, the “all students” group is fairly evenly divided between achieving at Levels II (35% to 40%) and III (30% to 35%).
Table 5 shows 5th grade performance on the reading MCAs in more detail. Table 5. 5th Grade MCA Reading
Performance
*All percentages
have been rounded The largest portion of LEP students (63% to 66%) score at Level I
while the largest portion of all students (37% to 41%) score at Level II. The second largest portion of LEP students score
at Level II (29% to 31%) while the second largest portion of all students score at Level
III (30% to 33%). The third largest portion
of LEP students (4%-5%) score at Level III while the third largest portion of all students
score at Level I (18-21%). Finally, the
smallest percentage of both groups (<1% and 8% to 11% respectively) fall in Level IV. Roughly 5% to 6% of LEP students at grade 5 score
in Levels III and IV on the Reading MCA and would be on track to complete the high
standards in high school if they continue to perform at the same level. In comparison, 38% to 44% of the “all
students” group appears to be on track to complete the high standards if they
continue to perform at the same levels. Discussion The data presented in this report are some of the first data
presented nationally on the performance of LEP students on a statewide assessment. Because the data represent only two years, it is
not yet possible to discuss trends. It is
important that information provided by the analyses presented here be used for system
accountability and to improve the instruction of LEP students, not to blame these students
for low performance levels. It is also
extremely important to continue to examine the participation and performance of LEP
students over time and to look for improvements resulting from specific types of
instructional programs. With these cautions in mind, the following points can be made: The participation of LEP
students in MCA math and reading tests at both grades 3 and 5 is high. Only about 6% to 11% of LEP students are not
participating. However, The performance of LEP
students in math at grades 3 and 5 looks different from the performance of those students
in reading at the same grades. The scores of LEP students and of all students in 3rd and 5th grade reading MCAs were fairly consistent in 1998 and 1999. Roughly 55% to 60% of LEP students in both grades
scored in Level I on the reading test in both years.
Roughly 18%-20% of the all students group at 3rd and 5th grade scored at Level I on the reading test in both years. Fifth grade math also shows that scores from
1998 to 1999 were fairly stable for LEP students with about 60% of them scoring in Level
I, and stable for all students with about 20% scoring at Level I. In contrast, there was much less consistency
between the percentages scoring in each level on the 3rd grade math MCA in 1998 and 1999.
This difference in achievement may indicate a difference in the test items
in either 1998 or 1999. The data in this report
cannot show us whether LEP students know the content of the math test but have difficulty
expressing their knowledge in English, or whether LEP students also lack the content
knowledge that is necessary for the math tests. It may be that LEP students who are in
pullout ESL programs are missing content instruction in math. Since individual schools and districts have
different models for providing ESL and Bilingual Education services, it would be
beneficial for schools and districts to do their own analyses of their LEP students’
MCA results and to relate these scores to the instruction students receive. Determining whether LEP students have the
opportunity to learn the content material and skills that are being assessed on the MCAs
is an important part of including these students in standards-based education. From the elementary
grades, LEP students need specific instruction in the academic English skills required on
this test (i.e., reading and writing). Students also need exposure to the types of
vocabulary used to describe mathematical processes (e.g., what was the mean score). To aid educators in determining whether LEP
students are making progress in academic English, CFL implemented a test of emerging
academic English reading and writing skills in the fall of 1999 that many LEP students
across the state took. Results of this test
should help educators better evaluate the impact of the instruction LEP students receive. If the LEP students in
this sample continue to perform at the same level as they did on the 1999 MCAs,
significant numbers of LEP students will have difficulty participating in the high
standards at the high school level and receiving a high school diploma. Further
investigation is needed to determine the impact of the accommodations offered to LEP
students taking the MCAs. There may be
accommodations that are not currently allowed, but that allow LEP students to better
demonstrate their content knowledge. In a
survey of English as a Second Language and Bilingual teachers, the Minnesota Assessment
Project found that these teachers would like to see dictionaries allowed as testing
accommodations for LEP students because dictionaries are a type of accommodation used in
daily instruction (Quest, Liu, & Thurlow, 1997). Continued collection of data, and refinements in the data
collected, will help the field to better understand the performance of LEP students. Eventually, the data will enable us to examine
the effects of instructional programs on these students’ performance. References Minnesota Department of Children, Families and Learning. (2000). 1998 Minnesota Comprehensive Assessments for Grades 3
& 5: Q & A. Available: http://cfl.state.mn.us/GRAD/Grade35/qa.html National Research Council. (1999).
Testing, teaching, and learning: A guide for states and school districts. Elmore, R., & Rothman, R. (Eds.). Washington, DC:
National Academy Press. Available: http://books.nap.edu Quest, C., Liu, K., and Thurlow, M. (1997, May). Cambodian,
Hmong, Lao, Spanish-Speaking, and Vietnamese parents and students speak out on
Minnesota’s Basic Standards Tests (Minnesota
Report 12). Minneapolis: University of Minnesota, National Center on
Educational Outcomes. United States Code Service. (1999). United States Code
Service—20 USCS section 6301. [On-line]. Available: http://web.lexis-nexis.com/congcomp/document?_ansset=GeHauKO-. Appendix A Minnesota Assessment Project Report Relating to Non English Language Background (NELB) and Limited English Proficient (LEP) StudentsReport 15: Educators’ Responses to LEP Student
Participation in the 1997 Basic Standards Testing Report 11: A Review of the Literature on Students with
Limited English Proficiency and Assessment Report 8: Resources: Limited
English Proficient Students in National and Statewide Assessments |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
