Prepared by Richard Spiccuza, Kristin Liu, Bonnie Swierzbin, John Bielinski, Martha Thurlow
Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:
Spiccuza, R., Liu, K., Szwierzbin, B., Bielinski, J., & Thurlow, M. (2000). Participation and performance of limited English proficient students during second attempts on a graduation exam (Minnesota Report No. 28). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/MnReport28.html
Over the past four years, the state of Minnesota has introduced a
standards-based system of accountability to reform Minnesota education statewide.
Beginning in 1996, 8th graders have been required to pass minimum competency tests in
order to receive a high school diploma. These Basic Standards Tests in reading and
mathematics, with the recent addition of a writing test, are administered to all students,
including those students with limited English proficiency (LEP), although certain
exemptions apply. In the past, LEP students across the nation often were excluded from
large-scale assessments or tested inappropriately (Abedi, Lord & Hofstetter, 1998;
Rivera & Vincent, 1996). When LEP students have been included in assessments, their
scores may not have been reported, either aggregated with other scores, or disaggregated
based on LEP statutes (August, Hakuta & Pompa, 1994).
Educators may have followed these testing practices in the belief
that LEP students would benefit. However, the end result may be that the educational needs
of LEP students are ignored because little is known about their academic progress. To
date, there is only a limited amount of data on LEP students’ participation and
performance in these types of assessments (Olson & Goldstein, 1997; O’Malley
& Valdez Pierce, 1994). There is even less longitudinal data tracking the
participation and performance of LEP students in large-scale assessments over time
although some initial data were reported by Liu and Thurlow (1999).
The Minnesota Assessment Project is a four-year federally funded
research project that examines the participation and performance of LEP students in the
Basic Standards Tests (BSTs). One of the goals of the project is to provide longitudinal
data on the test performance of LEP students, both those who receive English as a second
language services and those who do not, in order to begin to understand the consequences
of educational reform for these students.
Since the BSTs are high-stakes exams, students first participate
in the 8th grade and continue participating through the 12th grade until the tests are
passed. For the years discussed in this report, a minimum of 75% of the items in each test
had to be answered correctly in order for students to pass. To increase our understanding
of how LEP students perform over time on the BSTs, this report presents the performance of
those students who failed in the first round of testing for the school years 1996 through
1998. This study specifically addressed the following research question: For second-time
test takers, is there a range of first-time scores that predicts passing the test on the
second try?
Method
Data were accessed from databases maintained by the Minnesota
Department of Children, Families and Learning (CFL). Student BST scores were selected from
the CFL database whenever scores occurred for both 1997 and 1998. State student
identification numbers were used to match 1997 and 1998 test scores for those years of BST
testing selected for this study. Therefore, these analyses only include students in the
database with valid identification numbers in both years. These students are a subset of
the entire population of approximately 150,000 students in grades 8 through 12 who tested
in 1998. They should be not be considered representative of the entire population. For
example, students who took the BSTs for the first time in 1998 are not included in the
data in this report. In addition, even though those students who failed the BSTs the first
time are referred to as “second-time test takers” in this report, a few of these
students may be taking the test for the third time. For instance, if a student initially
failed as an 8th grader in 1996 and again as a 9th grader in 1997, the student would have
taken the test for the third time in 1998. Such students are not distinguished from
second-time test takers in this report.
The term “LEP” in this report is defined to mean those
students who were identified in the CFL databases as receiving English as a second
language (ESL) or bilingual education services.
Participation and Performance of Second-Time Test Takers
There were 19,534 students in grades 8 through 12 who took the BST Reading test in both 1997 and 1998 (Table 1); of these, 52% (n = 10,087) passed in 1998. On the other hand, for the 819 LEP students out of the 19,534 who were tested with the BST Reading test in 1997 and 1998, only 17% (n = 140) second-time test takers passed in 1998. While the numbers differ for the BST Mathematics test, the pattern is similar to the BST Reading Test; that is, a higher percentage of the set of all second-time test takers passed the test in 1998 than of the LEP second-time test takers (see Table 1).
Table
1. All Students and LEP Students in Grades 8-12 who participated in the Mathematics and
Reading BSTs in both 1997 and 1998
|
Students Testing in 1997 and in 1998 |
Students Failing in 1997 and Passing in 1998 |
Percent Passing in 1998 |
Reading |
|
|
|
All Students |
19,534 |
10,087 |
52% |
LEP Students |
819 |
140 |
17% |
Mathematics |
|
|
|
All Students |
14,566 |
5,076 |
35% |
LEP Students |
878 |
177 |
20% |
There were 14,566 students who took the BST Mathematics test in
both 1997 and 1998; of this number, 35% (n = 5,076) passed in 1998. Of the 878 LEP
students out of the larger set of 14,566 students, 20% (n = 177) passed in 1998. When all
students are considered together, 17% more passed the Reading Test the second time than
the Mathematics Test, but when only the LEP students are considered, 3% more students
passed the Mathematics Test the second time than the Reading Test.
In order to analyze the change in BST performance of second-time test takers, for each test we chose the population of “all students who failed in 1997” and divided these students into groups: seven groups for the Reading test and nine groups for the Mathematics test. Group divisions were made arbitrarily based on the distribution of scores. The first group of students for each test scored 1-25% of items correct; that is, these students achieved below or at the level usually designated as chance-level performance. The number of groups for the Mathematics test is larger than the number for the Reading test because a finer distinction among scores was desired for the Mathematics test where more students scored near the minimum passing score of 75%. Table 2 shows the range of scores that falls within each of the seven groups for reading and nine groups for mathematics. For purposes of comparison, then, LEP students were categorized into these same groups based upon their test performance in 1997. In this report the performance of all students and LEP students is discussed in the context of these groups.
Table
2. Groups of Students Based on the Performance of All Students Who Failed in 1997
Group |
Percent of Items Correct in Reading |
Percent of Items Correct in Mathematics |
1 |
1-25% |
1-25% |
2 |
26-35% |
26-35% |
3 |
36-45% |
36-43% |
4 |
46-55% |
44-50% |
5 |
56-63% |
51-54% |
6 |
64-68% |
55-59% |
7 |
69-74% |
60-65% |
8 |
N/A (passing) |
66-69% |
9 |
N/A (passing) |
70-74% |
The next series of tables, Tables 3 through 10, shows the
performance of “all students tested in 1997 and 1998” (including LEP students),
as well as the performance of only LEP students who initially failed the BSTs in 1997.
Table 3 displays the BST Reading performance for all students who failed the test in 1997 and passed in 1998; Table 4 displays the BST Reading performance for LEP students who failed the test in 1997 and passed in 1998. In each table, the score group for which approximately 50% of second-time test takers passed in 1998 is marked with a double asterisk (**). In the case of all students (Table 3), score group 5 is marked, indicating that more than one-half of all students (60.3%) who achieved 56-65% correct on the BST Reading test in 1997 the subsequent year. Students in groups 6 and 7 demonstrated even greater rates of passing in 1998. In contrast, LEP students needed to be in score group 6 to approach a 50% chance of passing. Slightly less than half (47.2%) of these LEP students who achieved 64-68% correct on the BST Reading test in 1997 passed the test the following year.
Score
Group |
Students Passing in 1998 |
Total Students in Group |
% Pass Reading (1998) |
1 |
23 |
454 |
5.1% |
2 |
85 |
1,411 |
6.0% |
3 |
477 |
2,516 |
19.0% |
4 |
1,565 |
4,014 |
39.0% |
**5 |
2,706 |
4,487 |
60.3% |
6 |
2,851 |
3,806 |
74.9% |
7 |
2,380 |
2,846 |
83.6% |
**Indicates the score group for which approximately 50% of second-time test takers passed.
Table
4. LEP students failing reading in 1997 and passing in 1998 by score group
Score
Group |
Students Passing in 1998 |
Total Students in Group |
% Pass Reading (1998) |
1 |
4 |
111 |
3.6% |
2 |
5 |
223 |
2.2% |
3 |
26 |
204 |
12.7% |
4 |
43 |
149 |
28.9% |
5 |
36 |
84 |
42.9% |
**6 |
17 |
36 |
47.2% |
7 |
9 |
12 |
75.0% |
**Indicates the score group for which approximately 50% of second-time test takers passed.
For both all students and LEP students, the percentage of students passing the Reading test in 1998 generally increased as the score group increased; however, for each score group, the percentage of LEP students passing in 1998 is lower than the percentage of all students passing in 1998. This difference is illustrated in Figure 1. Note that the difference is smallest (less than 7 percentage points) for the groups whose scores on the 1997 tests were the lowest (score groups 1 through 3); the difference is fairly small (8.6) between score group 7 for the sets of all students and LEP students, and the largest difference (27.7) is between score groups 6.
Figure 1. Percent Passing BST Reading Test in 1998 by Score Group for All Students and LEP Students
Tables 5 and 6 display the mean percent correct on the 1997 BST
Reading test for each score group and the mean percent correct on the 1998 BST Reading
test for the students who had been in a particular score group in 1997. For example, the
last row of Table 6 shows that LEP students in score group 7 had a mean of 72% on the 1997
BST Reading test. Those students achieved a mean of 77.8% on the 1998 BST Reading test.
The mean number of percentage points gained from 1997 to 1998 (in this case, 5.8) is shown
in the last column of each table and is illustrated in Figure 2. For the most part, the
general population of students outperformed and made larger gains than the LEP students
with similar 1997 test scores.
Table 5. Mean Percent Correct for All Students on BST Reading Test in 1997 and 1998
Score
Group |
Mean % Correct BST Reading 1997 |
Mean % Correct BST Reading 1998 |
Mean Gain 1997 to 1998 |
1 |
22.0 |
43.8 |
21.8 |
2 |
32.0 |
50.0 |
18.0 |
3 |
42.0 |
59.9 |
17.9 |
4 |
52.0 |
68.9 |
16.9 |
5 |
61.0 |
75.6 |
14.6 |
6 |
67.0 |
79.9 |
12.9 |
7 |
72.0 |
82.9 |
10.9 |
Table
6. Mean Percent Correct for LEP Students on BST Reading Test in 1997 and 1998
Score
Group |
Mean % Correct BST Reading 1997 |
Mean % Correct BST Reading 1998 |
Mean Gain 1997 to 1998 |
1 |
21.0 |
43.6 |
22.6 |
2 |
32.0 |
48.7 |
16.7 |
3 |
44.0 |
58.1 |
14.1 |
4 |
52.0 |
65.6 |
13.6 |
5 |
60.0 |
70.3 |
10.3 |
6 |
66.0 |
73.5 |
7.5 |
7 |
72.0 |
77.8 |
5.8 |
Figure 2. Mean Gain on BST Reading Test by Score Group for All Students and LEP Students
Score group 1 of the LEP students outgained score group 1 of all
students slightly (22.6 versus 21.8 percentage points); however, this was not true for any
other score groups. While the gain in percentage points from 1997 to 1998 declined for
both all students and LEP students as the score group increased, for the LEP students the
decline was more precipitous: from 22.6 for score group 1 to 5.8 percentage points for
score group 7. In contrast, the gain for the set of all students declined by one-half from
score group 1 (21.8) to score group 7 (10.9).
The BST Mathematics performance for all students who failed the
test in 1997 and passed in 1998 is displayed in Table 7. Following that is Table 8, which
displays the BST Mathematics performance for LEP students who failed the test in 1997 and
passed in 1998. In each table, the score group for which approximately 50% of second-time
test takers passed in 1998 is marked with a double asterisk (**). This is score group 8 in
each table, indicating that of students who scored 66-69% correct on the BST Mathematics
test in 1997, 55.0% of all students and 48.9% of LEP students passed in 1998.
Table
7. All students failing mathematics in 1997 and passing in 1998 by score group
Score
Group |
Students Passing in 1998 |
Total Students in Group |
% Pass Math (1998) |
1 |
4 |
101 |
4.0% |
2 |
5 |
446 |
1.1% |
3 |
13 |
940 |
1.4% |
4 |
85 |
1566 |
5.4% |
5 |
146 |
1450 |
10.1% |
6 |
349 |
1757 |
19.9% |
7 |
1119 |
3035 |
36.9% |
**8 |
1563 |
2840 |
55.0% |
9 |
1792 |
2431 |
73.7% |
Table
8. LEP students failing mathematics in 1997 and passing in 1998 by score group
Score
Group |
Students Passing in 1998 |
Total Students in Group |
% Pass Math (1998) |
1 |
1 |
32 |
3.1% |
2 |
1 |
82 |
1.2% |
3 |
1 |
140 |
0.7% |
4 |
9 |
162 |
5.6% |
5 |
16 |
113 |
14.2% |
6 |
29 |
95 |
30.5% |
7 |
53 |
129 |
41.1% |
**8 |
46 |
94 |
48.9% |
9 |
21 |
31 |
67.7% |
For both all students and LEP students, the percentage of students passing the Mathematics test in 1998 showed similar patterns. The percentage of students passing the test in 1998 was lower for score groups 2 and 3 than for score group 1, and for score group 4 through 9 the percentage passing increased as the score group increased. This pattern is illustrated in Figure 3, which compares the percentage of all second-time test takers and LEP second-time test takers passing the Mathematics test in 1998. In contrast to the Reading test, for some score groups (5, 6, and 7) a higher percentage of LEP students than all students passed the BST Mathematics test in 1998.
The mean percent correct on the 1997 BST Mathematics test for
each score group and the mean percent correct on the 1998 BST Mathematics test for the
students who had been in a particular score group in 1997 is displayed in the second part
of Tables 9 and 10.
Table
9. Mean Percent Correct for All Students on BST Mathematics Test in 1997 and 1998
Score
Group |
Mean % Correct BST Math 1997 |
Mean % Correct BST Math 1998 |
Mean Gain 1997 to 1998 |
1 |
21.0 |
32.3 |
11.3 |
2 |
32.0 |
35.4 |
3.4 |
3 |
40.0 |
43.0 |
3.0 |
4 |
47.0 |
52.5 |
5.5 |
5 |
53.0 |
59.2 |
6.2 |
6 |
57.0 |
63.9 |
6.9 |
7 |
63.0 |
70.1 |
7.1 |
8 |
68.0 |
74.4 |
6.4 |
9 |
72.0 |
78.9 |
6.9 |
Table
10. Mean Percent Correct for LEP Students on BST Mathematics Test in 1997 and 1998
Score
Group |
Mean % Correct BST Math 1997 |
Mean % Correct BST Math 1998 |
Mean Gain 1997 to 1998 |
1 |
22.0 |
27.6 |
5.6 |
2 |
32.0 |
37.3 |
5.3 |
3 |
40.0 |
43.2 |
3.2 |
4 |
47.0 |
52.8 |
5.8 |
5 |
53.0 |
60.6 |
7.6 |
6 |
57.0 |
65.2 |
8.2 |
7 |
63.0 |
70.9 |
7.9 |
8 |
68.0 |
72.8 |
4.8 |
9 |
72.0 |
76.8 |
4.8 |
The mean number of percentage points gained from 1997 to 1998 is shown in the last column of each table, and an illustration of the difference between the sets of all second-time test takers and LEP second-time test takers is given in Figure 4. The mean gain for all second-time test takers ranged from 11.3 percentage points (score group 1) to 3.0 percentage points (score group 3). For the LEP second-time test takers, the mean gain ranged from 8.2 percentage points (score group 6) to 3.2 percentage points (score group 3). A comparison of Figures 2 and 4 shows that the pattern of mean gain by score groups is different for Reading and Mathematics. The BST Reading test has a wider range of gains both for all students (21.8 to 10.9) and for LEP students (22.6 to 5.8) than the Mathematics test has for all students (11.3 to 3.0) and for LEP students (8.2 to 3.2). In addition, neither the mean gain of all students nor that of LEP students on the Mathematics test decreases by score group as it does on the Reading test. For all students on the Mathematics test, the mean gain is the highest (11.3 percentage points) for score group 1, drops steeply for score groups 2 and 3 (3.4 and 3.0, respectively), and then increases again and levels off for score groups 6 through 9 (a range of 6.4-7.1). For the LEP students on the Mathematics test, the mean gain is 5.6 percentage points for score group 1, dips to a minimum of 3.2 for score group 3, increases to a maximum of 8.2 for score group 6, and finally decreases again for score groups 8 and 9 (4.8).
Figure
4. Mean Gain on BST Mathematics Test by Score Group for All Students and LEP Students
In summary there are several overarching trends that emerge from
these analyses and that support a better understanding of the participation and
performance of LEP students in Basic Standards Testing. First, it should be noted that
these analyses are based on only two years of testing data. While early interpretations
can be drawn, these data are not enough to determine trends in the performance of LEP
students. This is true in part because the test was optional in 1996 and 1997. Additional
years of data will provide a more accurate determination of trends in LEP student
performance.
With this caution in mind, the following points can be made:
•
Between 1997 and 1998, 20% or fewer of LEP students who retook
the BSTs passed. This percentage of successful second-time test takers compares to 35% of
all students passing the Mathematics test and 52% of all students passing the Reading test
on the second attempt.
•
LEP students who scored below 64-68% of items correct on the
Reading test and below 66-69% of items correct on the Mathematics test in 1997 had less
than a 50% chance of passing the test in 1998.
•
There was a small number of students whose test scores were in
score group 1 (i.e., 25% correct or less) on the Reading or Mathematics test in 1997 and
who passed in 1998. This suggests a need to look in greater depth at who these students
are and what instructional supports they received for their achievement.
The effect of remedial programming is one of the important
factors in second time test achievement, but it was beyond the scope of the current study.
Minnesota schools are required by law to provide remediation for students who do not pass
the Basic Standards Tests; however, the type of remediation is not specified. Programs
range from pull-out sessions with individual tutoring to summer school programs with a
computerized curriculum. Various types of remedial programs offered in Minnesota are
discussed in Schleisman, Peterson, & Davison (2000). Students in some districts can
take the BST again immediately after summer school instead of waiting several months until
the next large-scale test administration. The type of remediation a student received and
its proximity to the test administration may play significant roles in whether a student
passes the BST on the second attempt regardless of the group in which the student’s
score fell the first time. For English language learners taking the BST Mathematics test,
Teelucksingh (2000) has shown a positive effect of using the Accelerated Math Program, a
computerized program for tracking student progress toward meeting particular mathematics
objectives. Further studies are needed to document the types of remedial programming
provided for LEP students and the effects of remediation on students’ test scores.
After promising remedial practices are identified, additional resources will be needed to
make those practices available in other schools.
Individual student characteristics may also be an important
factor in second time test achievement. Further analyses are needed to examine individual
student results to determine the characteristics of LEP students who make a greater than
average amount of gain on the BSTs from one test administration to another. In a study of
four schools with high LEP student populations, Liu, Albus, Thurlow, Bielinski, and
Spicuzza (2000) found that student characteristics such as length of time in the United
States and in U. S. schools, levels of native language or English proficiency, and the
number of school changes a student has experienced play a direct role in test achievement.
This type of information, combined with information on the type of remediation students
are receiving, can provide valuable insight for educators on ways to help LEP students
succeed on large-scale, high-stakes assessments.
References
Abedi, J., Lord, C., & Hofstetter, C. (1998). Impact of selected background variables
on students’ NAEP math performance [On-line]. Available CSE Technical Report 478:
http://www.cse.ucla.edu/CRESST/Reports/TECH478.PDF.
August, D., Hakuta, K., & Pompa, D. (1994). For all students: LEP students and Goals 2000. A
discussion paper. Paper presented at the NAE panel meeting No. 15., Washington, D.C.
Liu, K., Albus, D., Thurlow, M., Bielinski, J., & Spicuzza,
R. (2000). Factors related to the performance of LEP
students on Basic Standards Tests (Minnesota Report 27). Minneapolis, MN: University
of Minnesota, National Center on Educational Outcomes.
Liu, K., & Thurlow, M. (September, 1999). Limited English proficient students’ participation
and performance on statewide assessment: Minnesota Basic Standards Reading and Math,
1996-1998 (Minnesota Report 19). Minneapolis, MN: University of Minnesota, National
Center on Educational Outcomes.
Olson, J., & Goldstein, A. (May, 1997). The inclusion of students with disabilities and limited
English proficient students in large-scale assessments: A summary of recent progress.
Washington, D.C.: National Center for Education Statistics.
O’Malley, J. M., & Valdez Pierce, L. (1994). State
assessment policies, practices, and language minority students. Educational assessment , 2 (3), 213-255.
Rivera, C., & Vincent, C. (1996). High school graduation testing: Policies and practices
in the assessment of LEP students. Paper presented at the Council of Chief State
School Officers, Phoenix.
Schleisman, J., Peterson, K., & Davison, M. (2000). Back to the basics: An investigation of school- and
district-level remediation efforts associated with Minnesota’s basic standards for
high school graduation. Minneapolis: Office of Educational Accountability, University
of Minnesota.
Teelucksingh, E. & Ysseldyke, J. (2000). Enhancing the learning of English language learners:
Consultation and a curriculum based monitoring system (Classroom-Based Assessment
Project). Minneapolis, MN: University of Minnesota, National Center on Educational
Outcomes.