NCEO Report 413

Students with Disabilities in Educational Policy, Practice, and Professional Judgment: What Should We Expect?

Rachel F. Quenemoen and Martha L. Thurlow

May 2019

All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Quenemoen, R. F., & Thurlow, M. L. (2019). Students with disabilities in educational policy, practice, and professional judgment: What should we expect? (NCEO Report 413). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Table of Contents


Acknowledgments

We acknowledge Dr. Perry Zirkel, University Professor Emeritus of Education and Law at Lehigh University, for invaluable guidance to supplement our limited legal understanding and experience. His critical review of iterative drafts for the appropriateness of our assumptions, gaps or missing context, and accuracy of representation helped us correct the most egregious of mistakes while working outside of our field. The inferences we have made between educational and legal perspectives are vastly improved based on his advice. Any remaining errors and misconceptions are ours.

Executive Summary

A chain of interconnected events has occurred over the past four decades that can transform the lives of students with disabilities and their families. These events include the national policy shift to standards-based reform, legislative action that is reflected in reauthorizations of federal educational laws and regulations, and sustained state and local educational practice reforms. These reforms have resulted in state-by-state work to rethink and define what all students need to know and be able to do in order to be successful grade to grade, and ultimately, well-prepared for college and career. As part of a larger theory of action based on national policy and shifts in law, these reforms raised expectations for all students along with improved methods of measuring achievement for accountability purposes. State-led practice reforms have included, and in some cases focused on, rethinking how to ensure that students with disabilities are included in and benefit from these reforms.

This report offers a cross-disciplinary introduction to topics in educational policy, practice, and law that have highlighted critical questions related to expectations for students with disabilities. Within the field there have been disagreements about how to conceptualize the question of expectations and rights for students with disabilities, disagreements on how to interpret evidence or context for the larger field or for an individual, and disagreements on the path forward. This paper cannot solve these larger arguments, but can help foster shared knowledge among people with diverse perspectives. It is time to acknowledge and articulate what we know thus far about expectations for students with disabilities in the context of policy, practice, and evidence from implementation of reforms.

We address questions that are critical to a discussion of expectations held for students with disabilities. Then, we examine these from an educational perspective and a legal perspective. The three critical questions we address are:

  1. What evidence do we have that there are students with disabilities who cannot achieve to the same level expected for other students, even after appropriate, evidence-based instruction in the general curriculum based on state standards set for all students?
  2. If there is compelling evidence that some students cannot achieve, can educators agree on and reliably determine which students with disabilities cannot be expected to learn to the same level, and why, even after appropriate evidence-based instruction?
  3. If some students cannot be expected to learn to the same level, how can an appropriately ambitious but different standard of expectation be defined for them to ensure they are not ignored or excluded from benefits that other students are receiving from school accountability?

Finally, this report articulates lessons learned from state and local work on educational policy and practice, implications of the events of the past four decades for that work, and the subsequent understanding of the effects of that work from educational and legal contexts. The paper provides recommendations for action based on what thoughtful, informed professional judgment of appropriate educational opportunities for students with disabilities should be.


Overview

A chain of interconnected events has occurred over the past four decades that can transform the lives of students with disabilities and their families. These events include:

As part of a larger theory of action based on national policy and shifts in law, these reforms raised expectations for all students along with improved methods of measuring achievement for accountability purposes. State-led practice reforms have included, and in some cases focused on, rethinking how to ensure that students with disabilities are included in and benefit from these reforms.

The Individuals with Disabilities Education Act (IDEA) has been and is pivotal in ensuring that students with disabilities have access to and make progress in the general curriculum. This law has evolved over the past four decades since its initial enactment in 1975 as the Education of All Handicapped Children Act (PL 94–142). During those 40 years, there have been numerous court decisions about expectations for what IDEA calls a free appropriate public education (FAPE) for students with disabilities. These court decisions generally have deferred to professional judgment to determine what substantive outcomes should be expected for students with disabilities. Because of the courts’ reliance on professional judgment, it is critically important that state and local educators’ judgments reflect what we now know about appropriately challenging expectations for students with disabilities. For educators, there are two primary intervention points for improving outcomes of students with disabilities: (a) knowing what to expect, and (b) knowing how to support achievement based on those expectations.


Purpose of this Report

First, this report offers a cross-disciplinary introduction to topics in educational policy, practice, and law that have highlighted critical questions related to expectations for students with disabilities. Within the field there have been disagreements about how to conceptualize the question of expectations and rights for students with disabilities, disagreements on how to interpret evidence or context for the larger field or for an individual, and disagreements on the path forward. This paper cannot solve these larger arguments, but can help foster shared knowledge among people with diverse perspectives. It is time to acknowledge and articulate what we know thus far about expectations for students with disabilities in the context of policy, practice, and evidence from the implementation of reforms.

Second, this report offers specific lessons learned from 25 years of work with the U.S. Department of Education and with all 60 state education agencies that receive special education funds (i.e., 50 states plus other entities such as the District of Columbia, Guam, and Puerto Rico). We include lessons from states’ school- and university-based stakeholders and advocates. The information we consider here in order to understand the lessons learned about expectations focuses on two general groups of students with disabilities. First, we focus on those with the most significant cognitive disabilities (Quenemoen, 2009; Quenemoen, Kearns, Quenemoen, Flowers, & Kleinert, 2010; Thurlow, Lazarus, Larson, Albus, Liu, & Kwong, 2017). Second, we focus on students often characterized as persistently low performing (Almond, Quenemoen, Olsen, & Thurlow, 2000; Bechard & Godin, 2007; Lazarus, Thurlow, Ysseldyke, & Edwards, 2015; Thurlow, Lazarus, & Bechard, 2013). These two groups, which together account for about 30% of students with disabilities, are the ones that likely have been the most affected by expectations that are held for them, but all students with disabilities are at risk for the negative effects of pervasive and historical low expectations.

Finally, this report articulates implications of the events of the past four decades and the subsequent understanding of their effects from educational and legal contexts. This paper also provides recommendations for action based on what thoughtful, informed, professional judgment of appropriate educational opportunities for students with disabilities should be.

The Critical Questions

We address questions that are central to a discussion of expectations held for students with disabilities. Then, we examine these from an educational perspective and a legal perspective. The three critical questions we address are:

  1. What evidence do we have that there are students with disabilities who cannot achieve to the same level expected for other students, even after appropriate, evidence-based instruction in the general curriculum based on state standards set for all students?
  2. If there is compelling evidence that some students cannot achieve, can educators agree on and reliably determine which students with disabilities cannot be expected to learn to the same level, and why, even after appropriate evidence-based instruction?
  3. If some students cannot be expected to learn to the same level, how can an appropriately ambitious but different standard of expectation be defined for them to ensure they are not ignored or excluded from benefits that other students are receiving from school accountability?

First, we review the history of the debate in both education and law that provides context to these questions. Following a discussion of the historical context, we present findings from extensive policy work completed in every state as part of ESEA and IDEA assessment and accountability requirements. The findings relate to implementation of two ESEA and IDEA regulations that attempted to answer the question of who the students are who may need a different expectation for achievement, and how to define that different expectation for accountability purposes. We provide a brief overview of the regulations in question here.

First is what was commonly called the 1% Rule, which was codified in the ESEA regulations in 2004 and 2015 (Federal Register, 2002, 2003). The second was known as the 2% Rule, which was first codified (Federal Register, 2007) and later rescinded in regulations (Federal Register, 2015) and finally prohibited in ESEA legislation (2015). We describe states’ efforts to identify two separate groups in response to these Rules, including students with the most significant cognitive disabilities who may need an alternate but appropriately ambitious standard (i.e., the 1% Rule on alternate achievement standards), and a second group referred to as students in a “gray area,” persistently performing at lower levels from typical peers (i.e., the 2% Rule on modified achievement standards). The collective state experiences and findings in studying these two student groups will be used to show how the three critical questions have preliminary answers to the point of educational consensus over the past 25 years.


Educational Perspective on Expectations for Students with Disabilities

In the aftermath of the ESEA and IDEA reauthorizations, the question of what to expect from students with disabilities became central for educators implementing the law. The initial conversations focused on obtaining a better understanding of who students with disabilities are.

Early in these discussions, common misperceptions of who students with disabilities are emerged from policymakers and their stakeholders (e.g., parents, business partners, community members, adults with disabilities), educators, and the general public. Chief among them at the time was an erroneous assumption that most students with disabilities were students with mental retardation (the categorical label of mental retardation has been changed in federal law to intellectual disabilities, the term used in this paper). The label of intellectual disabilities is determined by state-set criteria in part based on low scores on intelligence tests (i.e., intelligence quotients or IQs).

In fact, students with intellectual disabilities at that time composed only 15% of the total population of students with disabilities ages 3–21 (U.S. Department of Education, 2002). The vast majority of students with disabilities, about 85%, did not have intellectual disabilities. Nevertheless, the national discussion continued to show a pervasive belief that students with disabilities overall, as well as the 15% of students with intellectual disabilities, should not be expected to learn the same grade-level content or the grade-level expectations as their peers. In 2004, Education Week (Olson, 2004) conducted a survey of special and general educators showing that 84% believed that students with disabilities should not be expected to learn what their peers were expected to learn.

In response to these perceptions, McGrew and Evans (2004) analyzed data from the nationally representative student sample used to norm the Woodcock-Johnson Battery, Third Edition (WJ-III, Woodcock, McGrew, & Mather, 2001). Using individual-level student data, McGrew and Evans examined the relationship between IQ and achievement. Given that the assumption that “most” students with disabilities have low IQs persists today, in the face of the fact that a small minority do, this report is worth revisiting, specifically addressing that small percentage of students with intellectual disabilities.

What to Expect of Students with Intellectual Disabilities (Cup Half Full or Half Empty?)

McGrew and Evans (2004) began their study by reviewing the debate about IQ test scores that started from the time these tests were first developed. They contrasted the work on intelligence of two contemporary early 20th century researchers—Alfred Binet with Sir Cyril Burt. They noted “Binet was an optimist who believed that the ability ‘glasses’ of children with lower ability were half full, and that their vessels could be filled further” (p. 3) with appropriate intervention and instruction. In contrast, they noted that

Burt’s work was based on the then popular view that intelligence was a genetically-based fixed entity. Burt’s ideas influenced the design of educational systems that segregated children in different educational tracks based on ability (p. 3).

Further, they cited Burt (1911) as saying “capacity must obviously limit content. It is impossible for a pint jug to hold more than a pint of milk; and it is equally impossible for a child’s educational attainments to rise higher than his educable capacity permits” (p. 3). These two philosophies endure today, framing the discussion of “what to expect.”

McGrew and Evans (2004) approached the debate from two directions. Using the nationally representative sample used for the WJ-III standardization (McGrew & Woodcock, 2001), they first explored the empirical relationship between intelligence and academic achievement. Given that the Binet-Burt glass-half-full/half-empty debate suggests that some students with low measured IQs may not be expected to learn, and thus not taught the content expected for their peers, McGrew and Evans also conducted a review of the literature on expectation effects. They summarized a large body of research on how what we expect a student to learn prior to teaching the student affects what they actually do learn.

What they found from analyzing the data from the WJ-III IQ and achievement data was “… even IQ tests that demonstrate some of the strongest correlations with achievement… cannot be used to provide perfect estimates of predicted achievement for individual students” (p. 9). For example, half of the students with IQs from 70 to 80, where expected achievement scores would be in the same range of 70–80, achieved above the expected level, with substantial numbers of them actually performing in the average achievement range. In other words, although there is a strong relationship between IQ and achievement when considering the entire group of students with low IQs, IQ scores alone cannot predict achievement well for any individual student, given the other factors that affect learning. McGrew and Evans concluded:

[I]ntelligence tests are fallible predictors of academic achievement. IQ test scores (and associated IQ-based disability category labels) are adequate, but not nearly sufficient metrics, by which to make reasonably precise predications about any particular individual student’s future expected achievement progress. It simply cannot be done beyond a reasonable doubt. The fallibility of IQ tests, coupled with the enduring presence of the ghost of Sir Cyril Burt’s deterministic IQ-achievement educational philosophy… raises the specter of many children with disabilities being denied the right to appropriate and demanding expectations. (p. 10, italics in original)

Effect of Teacher Expectations on Student Achievement

Although McGrew and Evans (2004) specifically addressed the reality for individual students among the 15% of students with disabilities who have intellectual disabilities, the challenge of appropriate and demanding expectations applies to all students with disabilities. To address the implications of the cup half full-half empty debate in classrooms and ultimate student achievement, McGrew and Evans relied on several meta-analyses on expectancy effects. They clarified that a link between expectations and IQ scores has not been found, but expectations and academic achievement are linked, with effect sizes that are small but statistically significant. Just as the understanding of implications for the group versus the individual seems small but meaningful, the implications for individuals affected by low expectations are life-changing. In addition, the effect of expectations accumulates over time. Year after year of lower (or higher) expectations for some students adds up to significant differences in outcomes. Further, one large-scale study (Madon, Jussim, & Eccles, 1997) found that the relative impact of expectations on achievement is greater for low achievers than for high achievers.

Although the effects of low expectations on an individual student are clearly harmful to that individual, it is also important to understand more about how groups are affected. McGrew and Evan’s (2004) literature review included a look at group-based stereotypes. They cited the work of Biernat (2003) extensively, who defined group-based stereotypes as “standards against which individual members of stereotype groups are judged” (p. 1019). McGrew and Evans summarized the effect in part as this:

when an individual (e.g., student with mental retardation) is a member of a group that is stereotyped as deficient on a trait or attribute (i.e., intelligence), evidentiary standards or expectations are often shifted in the direction of leniency, less challenge, and minimal competencies. (p. 27)

Later they concluded “… expectancy effects can be viewed as a form of standards-based stereotyping. This… can produce direct (assimilative) or indirect ‘hidden’ stereotyping effects, both of which can exert negative influence on academic performance” (p. 28). They cautioned that:

It is inappropriate to infer that the majority of educators are biased simply because they may hold differential expectations for some students…. Nevertheless, the literature raises numerous issues that are directly relevant to today’s educational context for students with disabilities in which both IDEA and ESEA are requiring improved performance. (pp. 28–29)

Overall, we have ample evidence that the students with disabilities “subgroup” as a whole, not just the small 15% with intellectual disabilities, have been affected by low expectations as individuals and as a group (e.g., Butrymowicz & Mader, 2017; Olson, 2004). In past decades, the debate about lowering of standards in the name of leniency, less challenge, and minimal competencies for this subgroup (Quenemoen, 2009; Lazarus et al., 2015; Minnema, Thurlow, & Scott, 2001) is part of the evidence. In the section of this paper on the search for a modified achievement standard, it will be clear that when states have had the opportunity to study persistently low-performing students with and without disabilities, what they found was that many lacked an opportunity to learn the content needed for successful learning, grade to grade.

The general consensus of these studies was that until students have been given the opportunity to learn, and the expectation they will learn, we risk doing harm by arbitrarily determining what can be considered “good enough” vs. “too much” as the expectation for most students with disabilities. That is true not only for the 15% whose IQ scores are low, but also for the 85% without intellectual disabilities.

Determining what desired outcomes should be for all students, as well as students with disabilities, is a complex discussion. The College and Career Readiness (CCR) debate in the early 2010s demonstrated that complexity (i.e., Camara & Quenemoen, 2012; Conley, 2010; Thurlow, 2014). Is CCR defined by preparation for college, career, or meaningful community/civic engagement? All three? At least one? For students with disabilities, particularly those with the most significant disabilities, is it the parent’s hope for “happiness” and avoiding dependency? Quality of life? Or do those aspirations equally apply to all children, all students? What should we expect?

In the field of severe disabilities, a critical component of current expectations and future aspirations is expressed as the “least dangerous assumption” (Donnellan, 1984). In the early days of implementation of IDEA, Donnellan proposed that until the field had data on what to expect from students with disabilities when they are given the opportunity to learn and appropriate educational services, supports, and specialized instruction on the content they need to be successful, we must assume they can learn it all. Unfortunately, we have limited data to confirm that students with disabilities have been given the opportunity to learn, and the expectation that they will learn (Elliott, 2015).

Criteria, Incidence, and Performance Data

If the glass is half full for the 15% with intellectual disabilities, what do we know about the other students with disabilities? For students whose learning may be affected by intellectual disabilities based on an IQ, as well as all other students who are identified as having a disability, the criteria used to determine disability status is highly subjective (Harry & Klingner, 2007), and varies considerably from state to state (GAO, 2019; Muller & Markowitz, 2004; U.S. Department of Education, 2018a). How state criteria are implemented at the local educational agency (LEA) and school level within a state is also highly subjective and variable. The use of specific criteria—and how the criteria are applied—also varies over time (Griffith, 2015; MacMillan, Gresham, & Bocian, 1998; MacMillan & Siperstein, 2002; U.S. Department of Education, 2018a).

Performance data for students with disabilities consistently show that on average students with disabilities perform below their peers without disabilities (Thurlow, Wu, Lazarus, & Ysseldyke, 2016; U.S. Department of Education, 2018b). Yet, the data also show that students with disabilities perform across the same range of scores as students without disabilities (see Figure 1 for grade 3 math performance in one state and Figure 2 for grade 3 reading performance in the same state).

As the state data reveal, making assumptions about what a student can learn based on that student having a disability is not warranted. In past decades, both ESEA and IDEA reauthorizations have emphasized the limitations of making any decisions on what to expect from any group of students or individual based on categorical labels. The legislation and related regulations have attempted to address competing concerns that perhaps “not all” students, including students with disabilities, may succeed on the state-defined achievement standards. Still, there continues to be limited clarity on which students those may be. A summary of these attempts is provided here, with a focus on specific efforts to define “which students can be expected to learn what content” in later sections.

Figure 1. State Data for Students with and without Disabilities in Grade 3 Math

Figure 1 Chart

Figure 2. State Data for Students with and without Disabilities in Grade 3 Reading

Figure 2 Chart

ESEA and IDEA Roles in Articulating Expectations

To understand the role of ESEA and IDEA in articulating expectations, it is necessary to understand a core element of the theory of action in standards-based reform, the role of state-defined standards (Elmore & Rothman, 1999). In standards-based reform, the standards include both content standards and achievement standards. Content standards define what students at each grade level should learn in reading, mathematics, science, social studies, or any content determined as necessary by the state and local education agency (LEA). Achievement standards are the specific state-defined performance expectations that show how well students will be able to perform on that content once they are given the opportunity to learn it. These achievement standards are used in each state’s accountability requirements, as an external check on how well all public schools are doing in educating all students.

In all states, the content standards are identified through collaborative work among content and instruction experts with broad-based stakeholder groups (e.g., citizens, parents, business representatives, advocates, educator organizations), who also advise on what expectation for performance on the content should be expected for students to be successful in the short and long-term. In all states, the actual achievement standards are set as part of developing statewide assessments of each content area in each grade, guided by requirements in ESEA and IDEA, but designed specifically to meet the needs and policy requirements of each state. These state-defined achievement standards are typically referenced to what the stakeholders have identified as desired outcomes, whether college, career, or community involvement, or broader goals of self-sufficiency and happiness. In most states, the state-defined content standards and achievement standards are the foundation of the general curriculum, although in most states, LEAs have substantial freedom in designing and implementing a curriculum based on the larger state-set standards that uniquely meets local preferences and needs.

ESEA and IDEA requirements do affect all state and local decisions about how each student is included in the standards; federal funding that flows to states and local districts and schools is dependent on adherence to these requirements, especially requirements related to standards-based assessments and accountability systems. Those requirements apply to all public schools, schools receiving federal dollars, and to all students being taught there, in order to ensure the quality of achievement data available to the public and to Congress for accountability purposes.

Specifically for students with disabilities, ESEA and IDEA requirements have addressed the possibility that for some students with disabilities, state-defined performance expectations—the achievement standards that are used to hold schools accountable for learning—may be defined differently for a small group of students who can be reliably identified using state-set guidelines and procedures. Even though the content standards on which the general curriculum is based and being taught to students in a specific enrolled grade will be the same for all students, for a small percentage of students with disabilities, the performance expectation of what they will learn in that content covered in the general curriculum can be adjusted, and the achievement standard used for system accountability also can be adjusted accordingly. It is up to each state to determine what an appropriate, challenging alternate expectation of learning is for that small group of students, and up to the LEA and school to decide how to teach them. In the case of all students with disabilities, the Individualized Education Program (IEP) team specifically designs the services, supports, and specialized instruction needed to ensure an individual student will have access to and make progress in the general curriculum, which is based on the state content standards. The IEP team also determines which of the state-set achievement standards is appropriately challenging for that student to gain meaningful educational benefit.

Before addressing the legal perspective on expectations for students with disabilities, here is a general summary of the ways students with disabilities currently are included in state-set expectations for achievement.

State-defined content standards and achievement standards that are the same as those set for nondisabled peers are for 85–90% of students with disabilities. At this time in ESEA and IDEA requirements, the vast majority of students with disabilities (85–90%) are expected to be taught and learn to the same expectations as their peers without disabilities (see Appendix A for source of this estimate). Just as for their peers without disabilities, not all students with disabilities will achieve at high levels, as demonstrated in Figures 1 and 2 on page 8. Students with disabilities and their peers without disabilities perform across the entire continuum of performance on state assessments measuring state-defined content and achievement standards. Both students with disabilities and those without disabilities will not perform well if they are not given the opportunity to learn the state-defined necessary content.

State-defined content standards for all students (same as above) adapted to reduce their depth, breadth, and complexity, on which states define an alternate achievement standard, commonly referred to as the “1%” alternate achievement standard, is for about approximately 10% of students with disabilities. For a very small group of students, defined as 1% of the total population of students, or about 10% of students with disabilities, alternate achievement standards are defined as what to expect after appropriate access to and progress in the general curriculum based on state-defined content standards. These students should not be identified by their categorical labels, which are subjective and vary across states and even across districts and schools within states (Griffith, 2015; MacMillan et al., 1998; MacMillan & Siperstein, 2002; U.S. Department of Education, 2018a).

We turn now to the debates on what to expect from which students that have occurred in case law over the past half a century. The core legal arguments have been about what a free appropriate public education (FAPE) described in IDEA should be, defined over time as four dimensions of FAPE, that is, procedural, substantive, failure to implement, and inability to implement (Zirkel, 2017a). In this paper, we focus primarily on the role of professional judgment related to the substantive dimension, that is, “Is the IEP reasonably calculated to enable the child to make progress appropriate in light of the child’s circumstances?” (Zirkel, 2017a, p. 3).


Legal Perspective on Expectations for Students with Disabilities

As described in the previous section, educators in every state have struggled to understand what to expect for students with disabilities as part of standards-based reform efforts. For most students with disabilities, state education agencies defined these expectations as the same standards each state set for all students. For a very small group of students with disabilities, no more than 1% of all students or approximately 10% of students with disabilities, a consensus has emerged based on both research and practice to identify and clarify expectations. The expectation for 1% was codified in the 2015 reauthorization of ESEA as the Every Student Succeeds Act (ESSA). That is, the small group of students defined in federal law as appropriately benefiting from state-set alternate achievement standards aligned to the state academic content standards are the one exception in all states to the requirement of the “same” standards for all students. Although educators have gradually come to this consensus, an intense legal debate also has occurred about expectations for these and other students with disabilities. We provide a brief summary of that debate as context.

The legal debate about expectations for students with disabilities began with the 1975 passage of the initial Education for All Handicapped Children Act (EHA, the predecessor version of the IDEA). During the intervening period, two major Supreme Court decisions have provided the frame for the discussion for the substantive floor of the FAPE requirement. This summary of the legal discussion is meant to provide grounding for educators and parents who need the basic outlines of the debate in order to understand the larger questions, arguments, and recommendations and to be able to act on them. Two Supreme Court decisions that are central to the legal debate on the substantive question of what to expect for students with disabilities are Board of Education of Hendrick Hudson Central School District v. Rowley (1982) and Endrew F. v. Douglas County School District RE-1 (2017). Each of these is described briefly.

Rowley (1982) Decision

In the first case, Amy Rowley was a deaf child who successfully completed kindergarten and progressed to first grade in a general education placement with an IEP that provided for: (a) an FM hearing aid, (b) a tutor for the deaf (one hour per day), and (c) speech therapy (three hours per week). Although Amy indisputably was making progress from grade to grade, her parents argued that under the IDEA requirement for FAPE she was entitled to an interpreter for her academic classes.

The Supreme Court decision did not agree with their position, noting the absence in the statute of any language supporting a substantive standard of “maximize[ing] the potential of handicapped children ‘commensurate with the opportunity provided to other children’” (Board of Education v. Rowley, p. 189). Instead, the Court enunciated the substantive side of judicial review as determining whether the IEP is “reasonably calculated to enable the child to receive educational benefits?” (p. 207). More specifically, for students with disabilities educated in regular classrooms, such as Amy Rowley, the Court formulated the minimum educational benefits as “achiev[ing] passing marks and advanc[ing] from grade to grade” (p. 204). Yet, noting the wide spectrum among students with disabilities from marginal to profound impairments, the Court warned: “We do not attempt today to establish any one test for [substantive FAPE],” instead limiting its analysis to a child with a disability “who is receiving substantial specialized instruction and related services, and who is performing above average in the regular classrooms of a public school system” (p. 202). Despite this caveat, the lower courts applied the substantive language of Rowley widely, with the only major distinction being with regard to the competing “some” and “meaningful” levels of benefits (Aron, 2005; Goetz, Pust, & Reilly, 2011; Goldschmit, 2011; Hannon, 1997; Johnson, 2012; Zirkel, 2017b).

Endrew F. (2017) Decision

The individual child in the more recent Supreme Court case was quite different from Amy Rowley. Diagnosed with autism at age 2, Endrew F. exhibited major behavioral problems leading to increased time in the special education classroom in grades 3 and 4. Dissatisfied, his parents unilaterally placed him in a private program for students with autism, where his behavior improved significantly and where he made the academic progress that had stalled in public school.

While acknowledging the caveat in Rowley, the Supreme Court in Endrew F. (2017) concluded that the earlier decision provides the basis for a general standard: “a school must offer an IEP reasonably calculated to enable a child to make progress appropriate in light of the child’s circumstances” (p. 999). For the requisite minimum level of progress for “most children” with disabilities, based on the Rowley expectation of their integration in regular classrooms, the Court cited the Rowley passing marks and grade-to-grade formulation, translating it as typically meaning “a level of instruction reasonably calculated to permit advancement through the regular curriculum” (p. 1000). Identifying Endrew F. as one of the remaining, relatively few students, the Court referred to a child who, unlike Amy Rowley, “is not fully integrated in the regular classroom and not able to achieve on grade level” (p. 1000). In such cases where grade-level advancement is not a reasonable prospect, the child’s IEP “must be appropriately ambitious in light of his circumstances, just as advancement from grade to grade is appropriately ambitious for most children in the regular classroom. . . .[including] challenging objectives” (p. 1000).

Emphasizing the individualized nature of IDEA, the Court declined to adopt a “bright line rule” for “appropriate progress,” instead, expressly conditioning it on the child’s individual circumstances. However, the Court repeated and refined the Rowley reminder of judicial deference to “the expertise and the exercise of judgment by school authorities,” specifying that “a reviewing court may fairly expect those authorities to be able to offer a cogent and responsive explanation for their decisions that shows the IEP is reasonably calculated to enable the child to make progress appropriate in light of his circumstances” (p. 1002).

From an educator’s perspective, the Court’s emphasis on integration in the regular classroom as a key to the child’s circumstance suggests an antiquated understanding of the implications of placement decisions. Placement decisions are understood by educators to reflect the least restrictive environment (LRE) in which a student can make progress in the general curriculum. The unique characteristics of the student may influence placement decisions, but those characteristics are not defined by placement decisions. Although Amy Rowley was fully integrated and Endrew F. was not, their placements are not the defining characteristic that distinguishes the two students educationally. In the context of the educator perspective, the description of Endrew F. suggests that he is most similar to those students for whom an alternate achievement expectation is warranted. The description of Amy Rowley suggests that she is most similar to those students for whom the general achievement expectation is warranted. For the purposes of this paper, that educational distinction between the characteristics of two students is used, not the legal reliance on the difference in placement in integrated settings.

Unresolved Legal Debates

These court cases are not a source for definitive answers to what professional educational judgment should reflect, but they do illuminate the legal debate that remains about what IDEA actually means for these two types of students. Both the Rowley and the Endrew F. cases addressed several questions that emerged from the original version of IDEA and continue in its successive reauthorizations, primarily concerning what constitutes FAPE, which is to be provided via the IEP. First, IDEA’s (2016) definition of FAPE is:

special education and related services that—(A) have been provided at public expense, under public supervision and direction, and without charge; (B) meet the standards of the State educational agency; (C) include an appropriate preschool, elementary school, or secondary school education in the State involved; and (D) are provided in conformity with the [IEP as specified in the Act]. (§1401[9])

IDEA also provides for the child’s placement to be in the least restrictive environment (§ 1412[a][5]).

Caveats on terminology. IDEA terminology is not necessarily understood in the same way in professional education and judicial contexts. For example, grade-to-grade progress in educational, standards-based reform specifically references the content and skills being taught and learned in each grade; legal discussions tend to use the term for promotion from one grade to another, regardless of knowledge and skills attained. Inclusive practices in education are on a continuum, from full inclusion in a general education setting, to majority time in general education settings, through higher proportions of time in separate settings, and ultimately to all time in fully segregated settings. Very few students with disabilities are on either end—fully included or fully segregated. Yet, as we have seen in the summaries of Rowley and Endrew F., in court decisions inclusion is sometimes described as all or none integration in the regular classroom. In addition, common educational definitions of the terms progress, ambitious, or benefit may not match up precisely with the use of the terms in law or court decisions. With those definitional cautions in mind, we share here examples of how expectations have been debated in legal contexts.

The legal issues that educators and legal advisors need to consider as context for defining appropriate expectations for a student are primarily related to a few core topics. Gleaned from a sample of multiple legal scholars across two decades (i.e., Aron, 2005; Dannenberg, 1997; Davison, 2016; Gill, 1996; Goetz et al., 2011; Goldschmidt, 2011; Hannon, 1997; Johnson, 2012; Kaufman & Blewett, 2012; Meredith & Underwood, 1995; Valentino, 2006; Yell, Conroy, Katsiyannis, & Conroy, 2013; Zirkel, 2017a, 2017b, 2017c, 2018), here are examples of the types of issues that appear to be unresolved.

FAPE calculation. What does it mean to receive FAPE? How much “progress” should be expected to result from instruction to meet the substantive standard for FAPE? How does “educational opportunity” fit into legal interpretations of IDEA? Just as the lower courts diverged in their “some” vs. “meaningful” interpretations of the Rowley benefit standard, they have already varied in their application of the refined Endrew F. progress standard.

Grade-to-grade progress. What does grade-to-grade progress mean relative to state established content and achievement standards? What is the difference between this term and the Endrew F. reference to “grade-level advancement”? Which students can reasonably be expected to make progress from grade to grade? Does inclusion in the classroom with students without disabilities affect the interpretation of what grade-to-grade progress means in the context of FAPE? How does that relate to the FAPE requirement of the least restrictive environment (LRE), which is referenced in the U.S. Department of Education definition of FAPE? Are the meanings ascribed to “mainstreaming” or “inclusion” the same in legal opinion and in educational practice?

For students like Endrew F. who may benefit from an alternate expectation to the full range of grade-to-grade progress, there is an additional set of questions:

Ambitious expectations. Where is the dividing line for students who are “not able to achieve on grade level”? What are “appropriately ambitious” and “challenging objectives” in terms of expected progress for this small subgroup of students? What external criteria could apply to LEA or school decisions on what an appropriately ambitious FAPE standard is for an individual student? How does inclusion in the classroom with peers without disabilities affect this determination?

For the larger community of educational and legal practitioners and scholars, the importance of understanding these questions in the context of what we have learned in educational practice since the late 1990s is critical. How can what we have learned about appropriately challenging expectations support the field in identifying what an appropriately challenging expectation is for the majority of students with disabilities, how a grade-to-grade progression is defined for these students, and how to identify the small number of students who may benefit from an appropriately ambitious standard on an alternate expectation? How, if at all, is the amount of time in an inclusive setting involved?

Answering the complex and nuanced legal questions well is beyond the scope of this paper. Given the importance of understanding the nature of the questions, however, we suggest the following resources as a starting point:

The educational questions are more focused, and the remainder of the paper addresses those. The core question for educators and families and the policymakers and administrators who guide them is: What should we expect of whom? We turn to that question in the next section.


What Should We Expect of Whom? Possible Answers

Across the states, policymakers and stakeholders have achieved substantial consensus on how to reliably identify the very small group of students—about 1% of the total population of students or approximately 10% of students with disabilities—who may benefit from an alternate achievement standard. There is emerging consensus on how to define that alternate achievement standard, although differences in both what and how well content is learned remain across state assessment programs. In contrast, policymakers and stakeholders have failed to reliably identify students, or define appropriate lower expectations, for the larger group of students with disabilities for whom modified standards were proposed and then rescinded—about 2% of the total population of students or 20% of students with disabilities (Cho & Kingston, 2011, 2015; Lazarus et al., 2015; Thurlow et al., 2013).

In this section we provide an overview of the consensus that has emerged on the approximately 1% of all students who may benefit from alternate achievement expectations. These are the students who, like Endrew F., appropriately may be taught to and learn based on different expectations than those for their typical peers. We also provide the results of several years of research that suggest we do not as yet have data to defend any other standards of performance for students other than those in the 1% group.

First, we provide a summary of how and when state content and achievement standards are developed and used. These are the definitions of what and how well students should learn to progress from grade to grade.

Content and Achievement Standards: Why They Matter In Defining Expectations

State-defined content and achievement standards are the foundation of statewide assessments used for accountability in standards-based reform. They define both what students should learn in each enrolled grade (content standards) and how well they should learn it (achievement standards). Assessments based on what and how well students should learn in each grade are used to hold the state and LEAs accountable for student learning.

Figure 3 shows the lengthy, multi-year process in which state policymakers and stakeholders define content standards that are the foundation for the general curriculum for all students, and achievement standards to determine how well students have learned the content. The connecting lines between steps show the sequence, going down the first column, up the second, and down the third column.

Figure 3. How and When States Define Content and Achievement Standards

Figure 3 Flow Chart

At two steps in the process, state standards are defined: the first box on the left column shows the starting point of Content Standards that are developed by policymakers and stakeholders. The first box in the right column shows the point when policymakers and stakeholders identify how well students should be expected to learn the required content, in the adoption of Achievement Standards. Content Standards are the same for all students, but the performance expectations developed based on that content follow two paths. The two paths are: (a) a general assessment path resulting in Achievement Standards that apply to most students with and without disabilities; and (b) an alternate assessment path that results in Alternate Achievement Standards that apply to those students with the most significant cognitive disabilities, about 1% of the total population of students (or approximately 10% of students with disabilities). This second group of students is the group for whom the Endrew F. decision is most likely to apply. State policymakers and stakeholders (e.g., educators, parents, business partners, community members, adults with disabilities) are involved at each major step to ensure the standards reflect the unique educational goals of each state and are appropriate for the state’s students.

The final step of this process is to hold state and local educators accountable for the performance of students compared to the “what and how well” that are defined in the content and achievement standards. It is this external check on “how good is good enough” that is the most important use of results from annual accountability assessments. The measurement results are designed to measure the educational system’s performance at state, district, and school levels. They yield high quality data on the collective level of performance and a general sense of whether an individual student is being appropriately challenged and instructed. Despite their importance for this purpose, annual accountability assessments should not be used to design IEPs because the tests were designed specifically for use at the systems level. Multiple and rich sources of information about the student other than state testing are used for IEP planning. Still, the use of the student’s test results do provide a general sense of whether the student is on the path to success on appropriately challenging objectives for his or her circumstances.

The determination of how a student with an identified disability or how groups of similar students are doing over time, compared to either the general or alternate achievement standards, can help determine whether progress being made is reasonably similar to what state stakeholders have defined as an appropriately challenging expectation for students at that grade level. Given that the Supreme Court has advised that the courts should show deference to professional judgment, but that jurists should expect to see evidence that deference is warranted, state-defined achievement standards provide an external check on that professional judgment. Low achievement for one student in one year does not flag a major systems issue, but a pattern of persistently low performance over multiple years, or low performance of all students in a particular category, would require investigation. Patterns of performance for students with disabilities can inform judgments of whether the overall system expectations and related services, supports, and specialized instruction are consistent with what the state has determined all students need for success in the next grade or in adult life.

State-defined criteria for participation in either general or alternate assessments also provide an external check on whether the appropriately challenging expectations are being held for any given student. For example, if a student has participated in alternate assessment but an external review of the decision-making process done as part of legal review shows that the student does not meet the state-set criteria for participation, then deference to professional judgment is called into question, and suggests inappropriately low expectations. Research documents inconsistent application of participation criteria. For example, Cho and Kingston (2011) studied data from two years of testing in one state and found that 5.3% of participants in a reading assessment based on modified achievement standards and 6.2% of students in a math assessment based on modified achievement standards had actually been proficient or above on the previous year’s general assessment. External review of the quality of decision making in any individual student’s case is one step in the review of professional judgment.

Because of its importance, we next discuss the current consensus on high quality decision-making criteria for participation in an alternate assessment. We highlight where some differences exist. Then we provide examples of what academic expectations for alternate achievement standards currently reflect.

Alternate Assessment Participation Criteria: The National Consensus

Brief history. Alternate assessments were first required in the reauthorization of IDEA in 1997. At that time, there was no specific definition of who might need alternate assessments other than for students “who cannot participate in State and district-wide assessment programs” as they were designed at that time. IDEA 1997 required that states and districts develop guidelines to identify the students. The purpose was to ensure that all students could be included in the public reporting of academic achievement for system accountability “with the same frequency and in the same detail as… the assessment of nondisabled children” (Sec 612(a)(17)(B)(ii–iii)). That inclusion was meant to ensure all students with disabilities benefited from standards-based reform under the theory of action for public accountability. In 2001, the ESEA reauthorization also required alternate assessments, and in 2003, ESEA regulations further defined alternate assessments:

[A]lternate assessments may be needed for students who have a broad variety of disability conditions; consequently, a State may employ more than one alternate assessment. An alternate assessment may be scored against grade-level standards, or, in the case of students with the most significant cognitive disabilities, against alternate achievement standards. Therefore, all students taking an alternate assessment are included in calculations of AYP [adequate yearly progress] as either proficient (and above) or non-proficient. (Federal Register, 2003, p. 68669)

The 2003 regulation clarified that a state could use an alternate assessment based on alternate achievement standards (AA-AAS) to measure achievement of students with the most significant cognitive disabilities as an equal contribution to other students in the required Adequate Yearly Progress (AYP) required by the law, with limits of 1.0% of the total student population counting as proficient or above. More than 1.0% of all students could participate in AA-AAS based on IEP team decisions, but a cap of 1.0% for accountability was put in place in part to avoid the unintended but demonstrated risk of students being put into an alternate assessment to avoid accountability for low performing students. (See Quenemoen, 2009, for a more complete history of the early development of AA-AAS.)

The 1% proficiency requirements stayed in place until the ESEA reauthorization in 2015, when the cap of 1.0% was applied to the state participation rate rather than for the rate of proficient students. This limitation applied to the state, but not to a district. At the time of the 2015 reauthorization, patterns of alternate assessment participation in some states showed persistent higher use of alternate assessment than national data suggested appropriate (Wu & Thurlow, 2017).

For this paper, we focus on how states defined the term “students with the most significant cognitive disabilities” for use in decision making at the local level. These are the students most like Endrew F. In most states, this group includes some but not all students addressed by McGrew and Evans (2004), and includes some students but not all with three primary categorical labels (i.e., Intellectual Disability, Autism, Multiple Disabilities) and a few students with almost all other categorical labels (Kearns, Towles-Reeves, Kleinert, Kleinert, & Thomas, 2011; Towles-Reeves et al., 2012; Towles-Reeves, Kearns, Kleinert, & Kleinert, 2009). Given the earlier discussion on the limited use of categorical labels in understanding a child’s individual circumstances, states have not used labels as a criterion for participation.

In 2016, a minority of states (N=17) had attempted to define significant cognitive disability with explicit, publicly available definitions (Thurlow et al., 2017). Most states, including those with a formal definition, had implemented a checklist approach to criteria, providing IEP teams with procedural guidance on how to consider multiple factors in determining whether a student can benefit from accountability with an alternate achievement standard (Thurlow et al. 2017). Most states found it necessary to focus on the guidance and training of decision makers to determine, on the basis of individual circumstances, who does and does not benefit from an alternate achievement standard (NCEO & CSAI, 2018). There is, in general, consensus on the criteria used as part of guidance about who participates in an alternate assessment.

National consensus and variations in participation criteria. In 2010, two consortia of states were funded by the U.S. Department of Education to develop a new generation of alternate assessments based on alternate achievement standards. They were the Dynamic Learning Maps (DLM) consortium and the National Center and State Collaborative (NCSC), later known as the Multi-State Alternate Assessment (MSAA). There were 31 states and unique entities (i.e., District of Columbia, Guam, Marianas, U.S. Virgin Islands) involved in the consortia at the time the two groups met in 2013 to discuss cooperation on definitions and participation criteria. The consortia states moved to rapid consensus on what should and what should not be considered by IEP teams when determining whether a student’s individual circumstances warranted participation in AA-AAS. Each consortium eventually fine-tuned the general agreement in their own procedures, including permitting each state to add additional guidelines as necessary, but by and large, this core consensus remained in the two consortia.

A comparison of the two groups’ participation criteria as of their 2017–2018 administration helps illustrate the degree of national consensus. Table 1 shows a side-by-side view of the participation criteria suggested by each consortium. They differed only by a few words.

Table 1. Comparison of DLM and MSAA Participation Criteria for 2017-2018

DLM MSAA
The student has a Significant Cognitive Disability. Review of student records indicate a disability or multiple disabilities that significantly impact intellectual functioning and adaptive behavior.
(Adaptive behavior is defined as behavior that is essential for someone to live independently and to function safely in daily life.)
The student has a significant cognitive disability. Review of student records indicate a disability or multiple disabilities that significantly impact intellectual functioning and adaptive behavior.
(Adaptive behavior is defined as essential for someone to live independently and to function safely in daily life.)
The student is primarily being instructed (or taught) using the EEs [Essential Elements] as content standards. Goals and instruction listed in the IEP for this student are linked to the enrolled grade level DLM EEs and address knowledge and skills that are appropriate and challenging for this student. The student is learning content linked to (derived from) state content standards. Goals and instruction listed in the IEP for this student are linked to the enrolled grade-level state standards and address knowledge and skills that are appropriate and challenging for this student.
The student requires extensive, direct, and individualized instruction and substantial supports to achieve measurable gains in the grade-and age-appropriate curriculum. The student requires extensive, repeated, and individualized instruction and support that is not of a temporary or transient nature; and
The student uses substantially adapted materials and individualized methods of accessing information in alternative ways to acquire, maintain, generalize, demonstrate, and transfer skills across multiple settings.
The student requires extensive, direct, individualized instruction and substantial supports to achieve measurable gains in the grade-and-age-appropriate curriculum. The student (a) requires extensive, repeated, individualized instruction and support that is not of a temporary or transient nature; and (b) uses substantially adapted materials and individualized methods of accessing information in alternative ways to acquire, maintain, generalize, demonstrate, and transfer skills across academic content.
Individual states may set additional eligibility criteria to establish which students are eligible to take the DLM alternate assessments. The criteria are the same across all states, as shared in the Test Administrator’s Manual (TAM). Guidance, however, might vary.

 

Each of the two consortia also provides a list of considerations that should NOT be used to determine participation, shown in Table 2. The lists are identical in content, with occasional reordering of the list in some participating states.

Table 2: Not Allowable Considerations for Determining Participation in Both DLM and MSAA

A disability category or label
Poor attendance or extended absences
Native language, social, cultural, or economic difference
Expected poor performance on the general education assessment
Academic and other services student receives
Educational environment or instructional setting
Percent of time receiving special education
English Language Learner (ELL) status
Low reading level/achievement level
Anticipated student’s disruptive behavior
Impact of student scores on accountability system
Administrator decision
Anticipated emotional duress
Need for accommodations (e.g., assistive technology/AAC) to participate in assessment process

 

In sum, these states are using virtually identical criteria to determine the appropriateness of participation in AA-AAS, based on individual student characteristics. Still, these criteria are not meant to be firm definitions. The complex characteristics of students challenge any label that could define them well; Endrew F. is a good example of the multiple and complex characteristics of these students.

The challenges of identifying the students helps inform the challenge of defining a percentage of students who can succeed at grade-level achievement, as discussed in the McGrew and Evans 2004 paper on expectations. Although the reauthorization of ESEA in 2015 limits participation in AA-AAS to 1.0% at the state level, at the local level there is ample room for variation across local education agencies, given the complexity of the students and the demography of districts within a state. The goal is to reduce the potential of harm for expecting either too little of students or expecting too much. Still, an articulated expectation of exactly what students who participate in AA-AAS should be expected to learn protects these students from the harm that comes from expecting nothing, which prior to the introduction of alternate achievement standards was often the case. The alternate achievement standard in law and education is to benefit the students, not harm them. In the next section, we turn to current state definitions of “how well” students should learn the content under an alternate achievement expectation.

Emerging National Consensus and Variations in Alternate Academic Achievement Standards

Achievement standards across the grades—and specifically the achievement level descriptors—define a reasonable expectation of grade-to-grade progress, an essential component in case law, including both the Rowley and Endrew F. Supreme Court decisions. We begin by reviewing how the two major general assessment consortia (PARCC and Smarter Balanced) used performance level descriptors (PLDs), also referred to as achievement level descriptors (ALDs), to define expected achievement. We provide examples from all four consortia of grade 4 expectations. Then, we reflect on how state and local leaders can use achievement standards to improve outcomes for all students; and how legal scholars and practitioners can use them to understand appropriate expectations for students with disabilities, through grade-to-grade progress.

A critical caution is in order. These examples are to demonstrate how to find, and appropriately make use of, state-by-state definitions of what should be expected at each grade level, whether the student is performing at a level of low, average, or high achievement. It is not meant to analyze the content chosen by states (in this case, the collective states in the four consortia) or make judgments about their choices. The choice of what content and what expectation is defined is up to each state, working through the process described above that involves state policymakers and stakeholders. For the purposes of this paper—sharing emerging understanding that will help educators, parents, and legal scholars and practitioners to determine appropriate expectations (and, by extension, whether FAPE has been provided)—the state-designed content and achievement standards appropriately are whatever the state has determined them to be. Those decisions are not being debated here.

General assessment performance level descriptors. A review of the purpose and use of achievement standards and PLDs/ALDs in general assessment (the assessment used with 99% of students, which includes 90% of students with disabilities) will set the scene for why they are so important in determining whether a student is making grade-to-grade progress. Here we focus on the statements of Smarter Balanced and PARCC on the purpose and use of their performance/achievement level descriptors. They use different terms to describe the level of detail in their PLDs/ALDs, but the excerpts provided here apply to the levels that describe grade-by-grade specific knowledge and skills.

Smarter Balanced describes four types of ALDs, including Policy, Range, Threshold, and Reporting, but we focus on those that can be used to understand the knowledge, skills, and process that students should be able to do at a grade level, as well as what they currently are able to do. The Smarter Balanced Range ALDs define content range and limits and are used in the development of items. Threshold ALDs define performance at the edge of the performance levels and are used for the standard-setting process. Reporting ALDs describe the knowledge, skills, and processes that test takers have as evidenced in the annual test administration. According to Smarter Balanced:

Reporting ALDs are the final ALDs that are developed following standard setting. They will provide guidance to stakeholders on how to interpret student performance on the test. These ALDs will be written after the standard setting in summer 2014. An important difference between the reporting ALDs and the range/threshold ALDs is that the reporting ALDs reflect student test performance. As such, they reflect the knowledge, skills, and processes that students can do. (Smarter Balanced, 2013a, b, p. iii)

Table 3 shows an example of a Grade 4 Range ALD for one target in one domain under one claim. There are four full pages of Smarter Balanced Grade 4 Range ALDs for mathematics, so Table 3 includes just a small sample of mathematics expectations at Grade 4 for the Smarter Balanced consortia states.

Table 3. Sample of Smarter Balanced Grade 4 Mathematics Expectations

RANGE ALD Grade 4 Mathematics (Claim 1, Domain 1 Concepts and Procedures, Operations and Algebraic Thinking, Target A: Use the four operations with whole numbers to solve problems). Each level assumes cumulative knowledge and skills listed in lower levels.
Level 1 students should be able to use the four operations (add, subtract, multiply, and divide) to solve one-step problems involving equal groups and arrays.
Level 2 students should be able to use the four operations to solve one-step problems involving an unknown number. They should be able to realize that it is appropriate to multiply or divide in order to solve familiar multiplicative comparison problems.
Level 3 students should be able to use the four operations (add, subtract, multiply, and divide) to solve one-step problems involving equal groups and arrays, including problems where the remainder must be interpreted. They should be able to find an unknown number and represent problems using equations with a symbol representing the unknown quantity.
Level 4 students should be able to assess the reasonableness of answers using mental computation and estimation strategies, including rounding.

Level 3 is considered “proficient” or “on track,” Levels 1-2 below expectations, Level 4 above expectations.

 

PARCC used an iterative development process with different grain size Performance Level Descriptors (which were called ALDs in Smarter Balanced). According to PARCC:

The Policy-Level PLDs… [are] used as a foundation for the development of PARCC’s grade- and subject-specific PLDs, which profile the knowledge, skills and practices students performing at a given performance level in a specific course or grade level are able to demonstrate. The grade- and subject-specific PLDs are an important tool for K-12 and postsecondary educators, parents, and students to gain a better understanding of the performance expectations for the PARCC assessments and how student mastery of the CCSS is evaluated through PARCC. The student assessment scores reported by performance level for schools and school districts for each grade and subject will also be important components of state accountability and public reporting systems, and numerous other state specific policies that use student performance results. (https://parcc-assessment.org/content/uploads/2017/11/PARCCCCRDPolicyandPLDsFINAL.pdf, p. 1)

Table 4 shows an example of a PARCC Reading Sub-Claim, which goes across reading literature, information, and vocabulary interpretation and use of evidence at each level. Each type of evidence is accompanied by a table linked to the PLDs, which provides quality of evidence necessary, accuracy of student evidence, and text complexity. Writing is included in separate descriptors. According to Patricia Fowler (email correspondence, July 30, 2018), “There isn’t a description for Level 1 in the PLDs because the PLDs describe the skills a student needs to ‘enter’ into each scoring category. Since 1 is the lowest score point, there is no description about what students need to do to gain entry to that score point.”

As the examples show, in both general assessment consortia, the general assessments are based on what state policymakers and stakeholders have agreed is the important knowledge and skills needed by students to advance from grade to grade. Reporting results on the annual state assessment can be used to understand how the system is doing, that is, to hold schools accountable for teaching students what they need for future success. These PLDs publicly document the knowledge and skills needed to successfully move from grade to grade, and are thus a rich resource for educators, parents, and legal scholars/judiciary.

In both the Rowley and Endrew F. decisions, conversations about grade-to-grade progress are central. Unlike the time when the Rowley decision was handed down, when promotion from grade to grade and academic grading systems defined reasonable progress, every state now has defined the actual knowledge and skills expected at each grade. At the time the Endrew F. decision was handed down, definitions were in place of actual knowledge and skills expected at each grade for students who participate in alternate assessment based on alternate achievement standards. However, as noted previously, the term “grade-to-grade” differs in legal usage compared to standards-based educational usage, and thus the actual legal implications are unclear. We turn to examples of those alternate achievement level descriptors next.

Table 4. PARCC Reading Sub-Claim

Grade 4 PARCC English Language Arts/Literacy – Examples of Reading Sub-Claim Perfor-mance Level Descriptors for Reading Literature, Reading Information, and Vocabulary Inter-pretation and Use

Level 2 partially meets expectations: In reading, the pattern exhibited by student responses indicates:

  • With very complex text, students demonstrate the inability to ask or answer questions, showing limited understanding of the text when referring to explicit details and examples in the text.
  • With moderately complex text, students demonstrate the ability to be minimally accurate when asking and/or answering questions, showing minimal understanding of the text when referring to explicit details and examples in the text.
  • With readily accessible text, students demonstrate the ability to be partially accurate when asking and/or answering questions, showing partial understanding of the text when referring to explicit details and examples in the text.

Level 3 approaches expectations: In reading, the pattern exhibited by student responses indicates:

  • With very complex text, students demonstrate the ability to be minimally accurate when asking and/or answering questions, showing minimal understanding of the text when referring to explicit details and examples in the text.
  • With moderately complex text, students demonstrate the ability to be generally accurate when asking and/or answering questions, showing basic understanding of the text when referring to explicit details and examples in the text.
  • With readily accessible text, students demonstrate the ability to be mostly accurate when asking and/or answering questions, showing understanding of the text when referring to explicit details and examples in the text.

Level 4 meets expectations: In reading, the pattern exhibited by student responses indicates:

  • With very complex text, students demonstrate the ability to be generally accurate when asking and/or answering questions, showing general understanding of the text when referring to explicit details and examples in the text.
  • With moderately complex text, students demonstrate the ability to be generally accurate when asking and/or answering questions, showing general understanding of the text when referring to explicit details and examples in the text.
  • With readily accessible text, students demonstrate the ability to be mostly accurate when asking and/or answering questions, showing understanding of the text when referring to explicit details and examples in the text.

Level 5 exceeds expectations: In reading, the pattern exhibited by student responses indicates:

  • With very complex text, students demonstrate the ability to be mostly accurate when asking and/or answering questions, showing understanding of the text when referring to explicit details and examples in the text.
  • With moderately complex text, students demonstrate the ability to be mostly accurate when asking and/or answering questions, showing understanding of the text when referring to explicit details and examples in the text.
  • With readily accessible text, students demonstrate the ability to be accurate when asking and/or answering questions, showing full understanding of the text when referring to explicit details and examples in the text.

 

Alternate assessment performance level descriptors. The alternate assessment consortia also defined what reasonable grade-to-grade progress entails. Examples of PLDs from both the DLM and the MSAA assessment consortia are included in this section.

When considering alternate achievement level descriptors, it is important to remember that they help us understand grade-to-grade expectations leading to success. As described in Figure 3, state policymakers and stakeholders have developed the PLDs. The PLDs are definitions of grade-to-grade expectations, and as such are essential resources in determining whether an individual student has received access—through specialized instruction and supports—to the same content and expectations as other students, whether based on general or alternate achievement expectations.

DLM states developed high level policy PLDs that defined the levels generally. After initial standard setting, grade and content-specific PLDs were developed from the work completed by standard-setting panelists as they evaluated profiles at each level. Starting with raw notes about critical skills and understandings for each performance level and the associated rationales, DLM test development content teams drafted PLDs for each grade and content area. These went through rounds of review and input from the partner states before they were finalized (Karvonen, Clark, & Nash, 2015, p. 58).

Table 5 shows an example of the Grade 4 DLM mathematics PLDs for the Year-End Model (that is, testing done at year’s end versus throughout the year, to be comparable to the examples from the other three consortia in this section). (See Appendix B for all Year-End grade 4 DLM ELA PLDs.) The example in Table 5 includes all the mathematics PLDs for the grade, unlike the partial general assessment consortia examples, which cover content at greater depth, breadth, and complexity.

MSAA states noted in the preface to their PLDs for ELA and Mathematics:

The performance descriptors… provide a detailed description for teachers, parents, and the public to see not only what grade-level content a student should know and be able to do in order to meet high expectations, but also the depth, breadth, and complexity of that content. (Multi-State Alternate Assessment, 2017, p. 147)

Table 6 shows the Grade 4 MSAA ELA PLDs (see Appendix C for the Grade 4 MSAA mathematics PLDs). Similar to the DLM example, the MSAA example includes all the ELA PLDs for the grade, unlike the partial general assessment consortia examples, which cover content at greater depth, breadth, and complexity.

These examples of grade 4 general and alternate PLDs for mathematics and for ELA illustrate the degree of detail that is available to determine what grade-level expectations are in any state. The PLDs are typically publicly available online, or by request, because they are an essential policy statement of each state’s approach to public education. An educator, parent, community member, or legal scholar or advocate can use these to determine whether a student has been provided access to the content-based instruction appropriate for his or her grade level, and to monitor grade-to-grade progress. That is, these can be used to determine whether students like Amy Rowley and students like Endrew F. have been provided access to the general curriculum defined by the standards for the state in which they are educated.

In the next section, we look at another group of students who are persistently low performing, the target group in a multi-year policy experiment of sorts to see whether they could: (a) be consistently identified, and (b) benefit from a modified achievement standard.

Table 5. DLM Grade 4 Mathematics—Performance Level Descriptors for Year-End Model

Emerging: A student who achieves at the emerging performance level typically looks for and makes use of mathematical structures (for example, patterns and attributes of shapes).

The student looks for and makes use of mathematical structures by

  • Attending to objects and shapes
  • Recognizing objects or shapes that are whole or in separate parts
  • Recognizing that a set is a group of objects or shapes with similar or different characteristics
  • Understanding the combining and dividing of objects by moving them to create a group or to create separate sets
  • Recognizing enclosures or boundaries and arranging objects or shapes into pairs based on attributes within the enclosure (for example, moving similar blocks into a box)
  • Combining or separating groups of objects to demonstrate the beginning concepts of addition and subtraction

Approaching the Target: A student who achieves at the approaching the target performance level typically identifies repeated calculations, calculates accurately, and attends to precision in computation and measurement.

The student identifies repeated calculations by

  • Solving repeated addition problems (for example, 2 + 2 + 2)

The student calculates accurately by

  • Adding and subtracting numbers within 20

The student attends to precision in computation and measurement by

  • Counting objects, ordering numbers, and classifying objects based on attributes
  • Communicating place value of numbers to the tens place
  • Recognizing patterns of numbers and symbols
  • Ordering numbers
  • Classifying objects based on attributes
  • Recognizing shapes divided into two or more parts
  • Recognizing math symbols (for example, symbols for lines, rays, and line segments)
  • Comparing the weight or volume of two objects
  • Identifying the value of coins (pennies, nickels, dimes, and quarters)
  • Recognizing the hour hand and minute hand on an analog clock
  • Recognizing hours and minutes on a digital clock

At Target: A student who achieves at the at target performance level typically calculates accurately, reasons abstractly, interprets data, and makes sense of problems and perseveres in solving them.

The student calculates accurately by

  • Adding or subtracting two-digit numbers up to 100
  • Rounding two-digit numbers to the nearest ten

The student reasons abstractly, interprets data, makes sense of problems, and perseveres in solving them by

  • Solving word problems with solutions up to 100
  • Identifying the core unit of a repeating number or symbol pattern (for example, in 123123123, the core unit is 123)
  • Recognizing parallel lines and intersecting lines in shapes
  • Comparing types of angles (for example, acute, obtuse, and right)
  • Counting unit squares to calculate area
  • Using appropriate tools (for example, scales, tiles, or measuring cups) to measure the weight, area, or volume of different objects
  • Identifying fractions up to one-fourth
  • Telling time to the hour and half hour on a digital or analog clock
  • Identifying coin names and values of coins (pennies, nickels, dimes, and quarters) and one dollar bills
  • Interpreting information on a graph and using that information to answer questions

Advanced: A student who is at the advanced performance level typically calculates accurately, reasons abstractly, explains reasoning, and uses appropriate tools to solve problems.

The student calculates accurately by

  • Adding or subtracting two-digit numbers with regrouping
  • Solving two-step addition or subtraction word problems
  • Multiplying numbers up to 12 by numbers 1 through 5

The student reasons abstractly and explains reasoning by

  • Rounding three-digit numbers to the nearest ten or hundred
  • Identifying the core unit of a repeating pattern (for example, in 123123123, the core unit is 123)
  • Extending a pattern that uses numbers or symbols
  • Comparing and ordering angles from largest to smallest or smallest to largest
  • Estimating the weight or volume of objects by comparing them to familiar objects in the environment
  • Calculating coin equivalency (for example, the number of nickels equal to one quarter)

The student uses appropriate tools to solve problems by

  • Telling time to the quarter hour on a digital and analog clock
  • Making predictions about data after interpreting a line graph

 

Table 6. MSAA Grade 4 ELA Performance Level Descriptors

Level 1: Low text complexityBrief text with straightforward ideas and relationships; short, simple sentences.

In reading, he/she is able to:

  • identify a topic of a literary text
  • identify a detail from a literary text
  • identify a character in a literary text
  • identify charts, graphs, diagrams, or timelines in an informational text
  • identify a topic of an informational text
  • use context to identify the meaning of multiple meaning words
  • identify general academic words

AND in writing, he/she is able to:

  • identify the concluding sentence in a short explanatory text
  • use the writing process to create a narrative product and demonstrate minimal (or no) com-mand of organization, idea development and/or conventions.

Level 2*: Low text complexity—Brief text with straightforward ideas and relationships; short, simple sentences.

In reading, he/she is able to:

  • determine the theme of literary text and identify supportive details
  • describe character traits using text-based details in literary text
  • determine the main idea of informational text
  • locate information in charts, graphs, diagrams, or timelines
  • use information from charts, graphs, diagrams, or timelines in informational text to answer ques-tions
  • use general academic words

AND with Moderate text complexityText with clear, complex ideas and relationships and simple compound sentences.

  • use details from a literary text to answer specific questions
  • use context to identify the meaning of multiple meaning words

AND with accuracy, he/she is able to:

  • identify simple words (i.e., words with a consonant at the beginning, a consonant at the end, and a short vowel in the middle)

AND in writing, he/she is able to:

  • • identify elements of a narrative text to include beginning, middle, and end
  • identify a concluding sentence related to information in explanatory text
  • use the writing process to create a narrative product and demonstrate limited command of organization, idea development and/or conventions.

Level 3*: Moderate text complexityText with clear, complex ideas and relationships and simple compound sentences.

In reading, he/she is able to:

  • determine the theme of literary text and identify supportive details
  • determine the main idea of informational text
  • explain how the information provided in charts, graphs, diagrams, or timelines contributes to an understanding of informational text
  • use information from charts, graphs, diagrams, or timelines in informational text to answer questions
  • use general academic words

AND with High text complexityText with detailed and implied complex ideas and relationships; a variety of sentence types, including phrases and transition words.

  • use details from a literary text to answer specific questions
  • describe character traits using text-based details in literary text
  • use context to identify the meaning of multiple meaning words

AND with accuracy, he/she is able to:

  • identify grade level words

AND in writing, he/she is able to:

  • identify a text feature (e.g., headings, charts, or diagrams) to present information in explanatory text
  • use the writing process to create a narrative product and demonstrate partial command of orga-nization, idea development and/or conventions

Level 4*: High text complexityText with detailed and implied complex ideas and relationships; a variety of sentence types including phrases and transition words.

In reading, he/she is able to:

  • determine the theme of literary text and identify supportive details
  • determine the main idea of informational text
  • explain how the information provided in charts, graphs, diagrams, or timelines contributes to an understanding of informational text
  • use information from charts, graphs, diagrams, or timelines in informational text to answer questions
  • use general academic words

AND in writing, students are able to do all skills listed in Levels 1-3 plus use the writing process to create a narrative product and demonstrate overall command of organization, idea development and/or conventions

From Multi-State Alternate Assessment (2017), pp. 146-161.

Note. Levels 2, 3, and 4 include demonstration of skills described in previous performance levels

 

The Search for a Modified Achievement Standard: The Rise and Fall of AA-MAS

In 2007, federal regulations were released permitting the development of modified achievement standards for up to 2% of students with disabilities, or what was called Alternate Assessment based on Modified Achievement Standards (AA-MAS). The regulations were targeted at—

[a] small group of students with disabilities whose progress is such that, even after receiving appropriate instruction, including special education and related services designed to address the students’ individual needs, the students’ individualized education program (IEP) teams are reasonably certain that the students will not achieve grade-level proficiency within the year covered by the students’ IEPs. (Federal Register, 2007, p. 17748)

In addition, the regulations required that the students would have an IEP that was aligned with grade-level academic content standards—that is, have evidence of access to the general curriculum for their grade level (Lazarus et al., 2015). After four years of state and researcher partnerships to identify who the students are and what challenging but appropriate expectation to define, the U.S. Department of Education announced it was moving away from this option. The Secretary of Education announced it this way at the American Association of People with Disabilities:

I want to say here and now for the record that we are moving away from the 2% rule. We will not issue another policy that allows districts to disguise the educational performance of 2% of students. That’s unacceptable, and that must change. We have to expect the very best from our students and to tell the truth about student performance so that we can give all students the supports and the services they need. (Diament, 2011)

After several years of supporting states as they transitioned away from use of AA-MAS, the U.S. Department of Education formally rescinded the option in August 2015 (Federal Register, 2015).

Lazarus et al. (2015) documented what happened in the intervening years as states grappled with defining the characteristics of students, identifying procedures to assign students to an appropriate testing option, and conducting research on changes to tests to reduce difficulty or barriers for some students, while maintaining the essential grade-level content. The primary conclusion they reached was that “states and districts have struggled with how to appropriately assign students to this test option, and that there is a need to ensure this group of students has access to rigorous standards-based content” (p. 1).

In addition to releasing the 2007 regulation, the U.S. Department of Education offered grants through two funding mechanisms to support research on the students, how they were instructed, and how best to assess them. Fourteen projects involving 26 states were funded. In a compilation of studies from all of the projects (Thurlow et al., 2013), a concluding chapter by Bechard, Lazarus, and Thurlow addressed primary lessons learned. Although the research questions across projects varied, and the findings were nuanced, three major points emerged from these lessons:

As shown in Figures 1 and 2, students performing at the lower end of the testing continuum include both students with and without disabilities. According to the 2% research done in multiple states, many of the students who repeatedly perform at the lowest levels, with or without disabilities, have not been given the opportunity to learn the state-defined content for their enrolled grade. There are very few students, with and without disabilities, who will achieve to the state-set achievement levels on the general curriculum based on state standards without being taught. Some may not achieve to expected levels even with the opportunity to learn, but the 2% research results suggested that we have not developed the ability—nor sufficient data to defend decisions—to determine which is which prior to teaching them all. The least dangerous assumption is to refocus on improving opportunities to learn for persistently low performing students, both those with and without disabilities.


Conclusions and Next Steps

As we reflect on what has been learned, we return to the questions we posed at the beginning of this review.

1. What evidence do we have that there are students with disabilities who cannot achieve to the same level expected for other students, even after appropriate, evidence-based instruction in the general curriculum based on state standards set for all students?

The 1% cap rule suggests that about 10% of students with disabilities, or 1% of the total population of students, can benefit from an alternate achievement standard. The 2% experiment suggested that until we ensure that all students are taught what is expected at grade level, with specialized instruction and supports as needed, we cannot make assumptions about what they can learn. Not all will succeed but to assume they will likely fail means that they will fail. This is the principled educational logic to which educators and the courts need to show deference, not traditional practices or budget-driven administrative decisions. A state-defined and articulated expectation of what students SHOULD be expected to learn protects students from harm.

2. If there is compelling evidence that some students cannot achieve, can educators agree on and reliably determine which students with disabilities cannot be expected to learn to the same level, and why, even after appropriate evidence-based instruction?

Comparing and contrasting participation requirements across two funded AA-AAS consortia, we find that these states are using virtually identical criteria to determine the appropriateness of participation in AA-AAS, based on individual student characteristics. These are guidelines and not firm definitions, given the complex characteristics of students similar to Endrew F. The reauthorization of ESEA in 2015 acknowledges this estimated percentage, and limits participation in AA-AAS to 1.0% at the state level, while providing ample room for variation across local education agencies. Thus, there is consensus on how to identify the small group of students who can benefit from an alternate achievement standard (around 1% of all students, 10% of students with disabilities).

On the other hand, 14 projects on the AA-MAS involving 26 states highlighted three major lessons (Thurlow et al., 2013): (a) educator perceptions of who the students were varied widely; (b) most of the students identified for the AA-MAS had not received adequate instruction; and (c) those AA-MAS that were developed by states did not result in significantly more students proficient (due to instruction issues). The several years of funded research on expectations for the additional 2% of students allowed to be held to a different standard through the AA-MAS failed to come to consensus on who the students are or how to reliably identify additional students for whom the benefits of a different achievement standard would outweigh the risk of harm.

3. If some students cannot be expected to learn to the same level, how can an appropriately ambitious but different standard of expectation be defined for them to ensure they are not ignored or excluded from benefits that other students are receiving from school accountability?

Setting alternate achievement standards should be to benefit the students, not harm them. A state defined and articulated expectation of what students SHOULD be expected to learn protects students from harm—the Amy Rowleys and lower performing (e.g., 2%) students held to general achievement standards, and the Endrew F.s held to alternate achievement standards.

Every state has developed policy definitions of what students should know and be able to do as part of their state and federally required accountability systems for both general and alternate expectations. These were developed through the lengthy, stakeholder-involved processes shown in Figure 3 and described in this paper. These articulated expectations protect all students with disabilities, especially students with significant cognitive disabilities, from the harm that comes from expecting nothing, which prior to the introduction of alternate achievement standards was often the case. These state policy statements are publicly available, either online or by request. An educator, parent, community member, advocate, or legal scholar can use them to determine whether a student has been provided access to the content-based instruction appropriate for his or her grade level, whether on general or alternate achievement, and to monitor grade-to-grade progress. That is, state performance level descriptors can be used to determine whether any student has been provided access to the general curriculum defined by the standards for the state in which the student is educated.

Next Steps. So what next? The processes and policies described here remain challenging to use, even after two decades of developing challenging but attainable grade-to-grade expectations. The processes and procedures are deeply entrenched in professional jargon and insider skills and knowledge. They seem—and probably are—overwhelmingly complicated and inaccessible to most policymakers, educators, advocates, and parents. What are some strategies that will turn these hard-won statements of high yet attainable expectations into powerful opportunities to improve outcomes for students with disabilities?

The transformative power of the reforms of the past four decades will be achieved only if today’s education professionals, advocacy groups and parents, special education policy organizations, and legislators have the opportunity to reflect on and act across disciplines and perspectives to understand what we have learned about expectations for students with disabilities. These educational and advocacy leaders can drive changes in the opportunities to learn and the academic achievement of students with disabilities. Lawyers and jurists will follow, not lead, these changes.

The most effective starting point may be discussions with state policymakers. They generally have the combination of skills, knowledge, and authority to leverage the power of the policies they have in place and the ability to reach all stakeholders with a common message. They will need to focus on how state oversight through existing accountability requirements in IDEA and in ESSA can create incentives and opportunities for educators at the local level to address needs. These incentives and opportunities should include directly addressing and documenting the effect of expectations held for students that do not match what is required at each grade, as defined in the stakeholder-defined content and achievement standards. State leadership should result in local educators committing to the same outcomes.

The avenues to increasing expectations and ultimately achievement will require systematic commitment to reducing the percentages of persistently low performing students with and without disabilities, improving access to the challenging grade-level curriculum, and high quality instruction, with a combination of acceleration and remediation for students far behind. These and other best practice and evidence-based strategies are not implemented well or fully in most schools. Ensuring that the state policy goals are translated into local implementation should include every teacher, including special educators who may serve as case managers for individual students. If teachers do not understand what expectations are for every student, and their responsibility to keep those expectations high, then all of this work is futile.

To do this effectively, parents and advocates will need support to understand and make use of what the state has determined all students need to be successful, and to leverage that work for individual students. State and local educators can partner with parent and advocacy networks, both state-supported and grass-roots networks, to build common understanding. Most parents are on a steep learning curve as they grapple with the effects of their child’s learning strengths and barriers. The wisdom they obtain through their child’s public school years is often lost to the system once the child graduates or exits. Leveraging that wisdom to systematically support new parents in initial understanding of what an appropriately high expectation can be is essential to ensure that they can advocate appropriately within the school. Helping parents understand what their child needs for future success can lead to parent and student understanding that students with disabilities will need to work harder than other students to overcome the effects of their disabilities. That is, parents are a critical link in ensuring students with disabilities are not enabled to use their disability as an excuse for easier options and low performance. Still, if educators who work with families do not understand what expectations should be when the student is taught well and does the work to achieve the expectations, then families almost certainly will not understand them.

The hope is that over time, all of these stakeholders will learn to make use of these state-by-state determinations of what grade-to-grade progress entails for students like Amy Rowley, AND a rigorous but alternate standard for students like Endrew F. By putting these policy definitions of expected performance into practice, we can eventually assist jurists as they trust but verify that professional judgment has resulted in an appropriately ambitious expectation for each individual student.


References

Almond, P., Quenemoen, R., Olsen, K., & Thurlow, M. (2000). Gray areas of assessment systems (Synthesis Report 32). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Aron, L. (2005). Too much or not enough: How have the circuit courts denied a free appropriate public education after Rowley? Suffolk University Law Review, 39(1), 1-25.

Bechard, S., & Godin, K. (2007). Who are the students in the assessment gaps, what are their attributes, and how do they perform? (Grant CFDA #84.368 of the U.S. Department of Education, Office of Elementary and Secondary Education, awarded to the Rhode Island Department of Education). Retrieved from http://www.necompact.org/full_research.htm#who

Bechard, S., Lazarus, S. S., & Thurlow, M. L. (2013). Lessons learned across projects for instruction and assessment. In M. L. Thurlow, S. S. Lazarus, & S. Bechard (Eds.). Lessons learned in federally funded projects that can improve the instruction and assessment of low performing students with disabilities (pp. 417-431). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Biernat, M. (2003). Toward a broader view of social stereotyping. American Psychologist, 58(12), 1019-1027.

Board of Education of Hendrick Hudson School District v. Rowley, 458 U.S. 176 (1982).

Burt, C. (1911). Experimental tests of higher mental processes and their relation to general intelligence. Journal of Experimental Pedagogy and Training, 1, 93-112.

Butrymowicz, S., & Mader, J. (2017, Nov. 6). Almost all students with disabilities are capable of graduating. Here’s why they don’t. Hechinger Report. Retrieved from https://www.huffingtonpost.com/entry/special-education-series_us_59fb588ae4b0415a420a55a6?88

Camara, W. J. & Quenemoen, R. F. (2012). Defining and measuring college and career readiness and informing the development of performance level descriptors. Retrieved from https://parcc-assessment.org/content/uploads/2017/11/Defining-Measuring-CCR-Camara-Quenemoen.pdf

Cho, H. J., & Kingston, N. (2011). Capturing implicit policy from NCLB test type assignments of students with disabilities. Exceptional Children, 78, 58-72.

Cho, H. J., & Kingston, N. (2015). Examining teachers’ decisions on test-type assignment for statewide assessments. Journal of Special Education, 49(1), 16-27.

Conley, D. (2010). College and career ready: Helping all students succeed beyond high school. San Francisco: Jossey-Bass.

Dannenberg, M. (1997). A derivative right to education: How standards-based education reform redefines the Individuals with Disabilities Education Act. Yale Law & Policy Review, 15(2), 629-654.

Davison, L. (2016). Endrew F. ex rel. Joseph F. v. Douglas County School Dist. Re-1: A missed opportunity (Student Paper). Retrieved from http://www.denverlawreview.org/dlr-onlinearticle/2016/12/19/endrew-f-ex-rel-joseph-f-v-douglas-county-school-dist-re-1-a.html

Diament, M. (2011, March 16). Duncan calls for end to inflated special education test scores. Disability Scoop. Retrieved from https://www.disabilityscoop.com/2011/03/16/duncan-sped-test-scores/12593/

Donnellan, A. M. (1984). The criterion of the least dangerous assumption. Behavioral Disorders, 9(2), 141-50.

Elementary and Secondary Education Act (ESEA) of 1994, 20 U.S.C. §§ 6301 et seq. (1994).

Elementary and Secondary Education Act (ESEA) of 2001, 20 U.S.C. §§ 6301 et seq. (1994).

Elementary and Secondary Education Act (ESEA) of 2015, 20 U.S.C. §§ 6301 et seq. (1994).

Elliott, S. N. (2015). Measuring opportunity to learn and achievement growth: Key research issues with implications for the effective education of all students. Remedial and Special Education, 36(1), 58-64.

Elmore, R. F, & Rothman, R. (Eds.). (1999). Testing, teaching, and learning: A guide for states and school districts. Washington, DC: National Academy Press.

Endrew F. v. Douglas County School District RE-1, 137 S. Ct. 988 (2017).

Federal Register. (2002, August 6). Title I — Improving the academic achievement of the disadvantaged, proposed rule, 67(151), pp. 50985-51027. Retrieved from https://www2.ed.gov/legislation/FedRegister/proprule/2002-3/080602a.html

Federal Register. (2003, December 2). Title I — Improving the academic achievement of the disadvantaged, final regulations, 68(236), pp. 68698-68708.

Federal Register. (2007, April 9). Title I — Improving the academic achievement of the disadvantaged: Individuals with Disabilities Education Act (IDEA), Final Rule 34 CFR Parts 200 and 300, 72(67), pp. 17748-17781. Retrieved from https://www.gpo.gov/fdsys/pkg/FR-2007-04-09/pdf/07-1700.pdf

Federal Register. (2015, August 21). Improving the academic achievement of the disadvantaged; Assistance to states for the education of children with disabilities, final regulations, 80(162), pp. 50773-50785. Retrieved from https://www.gpo.gov/fdsys/pkg/FR-2015-08-21/pdf/2015-20736.pdf

GAO. (2019). Special education: Varied state criteria may contribute to differences in percentages of children served (GAO-19-348). Washington, DC: U.S. Government Accountability Office.

Gill, S. L. (1996). Punitive damages: Flying in the face of the Individuals with Disabilities Education Act. Dickinson Law Review, 100, 383-413. Retrieved from https://dickinsonlaw.psu.edu/lawreview.

Goetz, A. J., Pust, T. L., & Reilly, A. (2011). The devolution of the Rowley standard in the Eighth Circuit: Protecting the right to a free and appropriate public education by advocating for standards-based IEPs. Hamline Law Review, 34, 503-530.

Goldschmidt, S. (2011). A new idea for special education law: Resolving the appropriate educational benefit circuit split and ensuring a meaningful education for students with disabilities. Catholic University Law Review, 60, 749-778.

Griffith, M. (2015). A look at funding for students with disabilities. The Progress of Education Reform, 16(1). Retrieved from https://www.ecs.org/clearinghouse/01/17/72/11772.pdf

Hannon, R. C. (1997). Returning to the true goal of the Individuals with Disabilities Education Act: Self-sufficiency. Vanderbilt Law Review, 50, 715-750.

Harry, B., & Klingner, J. (2007). Discarding the deficit model. Educational Leadership, 64(5), 16-21.

Individuals with Disabilities Education Act (IDEA) of 1997, 20 U.S.C. §§ 1400 et seq. (1997).

Individuals with Disabilities Education Act (IDEA) of 2004, 20 U.S.C. §§ 1400 et seq. (2004).

Johnson, S. F. (2012). Rowley forever more? A call for clarity and change. Journal of Law & Education, 41(1), 25-44.

Kaufman, A. K., & Blewett, E. (2012). When good enough is no longer good enough: How the high stakes nature of the No Child Left Behind Act supplanted the Rowley definition of a free appropriate public education. Journal of Law & Education, 41(1), 5-23.

Karvonen, M., Clark, A., & Nash, B. (2015). 2015 year-end model standard setting: English language arts and mathematics (Technical Report No. 15-03). Lawrence, KS: University of Kansas, Center for Educational Testing and Evaluation. Retrieved from https://dynamiclearningmaps.org/sites/default/files/documents/publication/Standard_Setting_Tech_Report_YE.pdf

Kearns, J., Towles-Reeves, E., Kleinert, H., Kleinert, J., & Thomas, M. (2011). Characteristics of and implications for students participating in alternate assessments based on alternate academic achievement standards. Journal of Special Education, 45(1), 3-14.

Lazarus, S. S., Thurlow, M. L., Ysseldyke, J. E., & Edwards, L. M. (2015). An analysis of the rise and fall of the AA-MAS policy. Journal of Special Education, 48(4), 231-242.

MacMillan, D. L., Gresham, F. M., & Bocian, K. M. (1998). Current plight of borderline students: Where do they belong? Education and Training in Mental Retardation, 33, 83-94.

MacMillan, D. L., & Siperstein, G. N. (2002). Learning disabilities as operationalized by the schools. In R. Bradley, L. Danielson, & D. P. Hallahan (Eds.) Identification of learning disabilities: Research to practice (pp. 287-368). Mahwah, NJ: Lawrence Erlbaum.

Madon, S., Jussim, L., & Eccles, J. (1997). In search of the powerful self-fulfilling prophecy. Journal of Personality & Social Psychology, 72, 791-809.

McGrew, K. S., & Evans, J. (2004). Expectations for students with cognitive disabilities: Is the cup half empty or half full? Can the cup flow over? (Synthesis Report 55). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

McGrew, K. S., & Woodcock, R. W. (2001). Woodcock-Johnson III technical manual. Itasca, IL: Riverside Publishing.

Meredith, B., & Underwood, J. (1995). Irreconcilable differences? Defining the rising conflict between regular and special educators. Journal of Law & Education, 24(2), 195-226.

Minnema, J., Thurlow, M., & Scott, J. (2001). Testing students out of level in large-scale assessments: What states perceive and believe (Out-of-Level Testing Project Report 5). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved from https://nceo.info/Resources/publications/OnlinePubs/OOLT5.html

Muller, E., & Markowitz, J. (2004). Disability categories: State terminology, definitions and eligibility criteria. Alexandria, VA: NASDSE, Project Form.

Multi-State Alternate Assessment. (2017). Multi-State Alternate Assessment 2015-16 technical report. Retrieved from https://cms.azed.gov/home/GetDocumentFile?id=58dc188f1130c01500d4b0c5#page127

National Governors’ Association. (1986). Time for results: The governors’ 1991 report. Washington, DC: Author.

NCEO & CSAI (2018). Proceedings of the 1% Cap National Convening: Supporting states in implementing ESSA’s 1% state-level cap on participation of students in the AA-AAAS. Minneapolis, MN: National Center on Educational Outcomes and Center for Standards and Assessment Implementation. Retrieved from https://rtc3.umn.edu/docs/OnlinePubs/NCEO_CSAI_ CapProceedings_508compliance_042619.pdf

Olson, L. (Ed.) (2004, January 8). Special education in an era of standards: Count me in (Special Issue). Education Week, 23(17). Retrieved from https://www.edweek.org/media/ew/qc/archives/QC04full.pdf

Quenemoen, R. F. (2009). The long and winding road of alternate assessments: Where we started, where we are now, and the road ahead. In W. D. Schafer & R. W. Lissitz (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 127-153). Baltimore: Brookes.

Quenemoen, R. F., Kearns, J., Quenemoen, M., Flowers, C., & Kleinert, H. (2010). Common misperceptions and research-based recommendations for alternate assessment based on alternate achievement standards (Synthesis Report 73). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Olson, L. (2004). Enveloping expectations. Education Week (Quality Counts), 23(17), 8-21.

Smarter Balanced. (2013a, April 26). Initial achievement level descriptors and collect content-readiness policy (English language arts/literacy). Retrieved from https://portal.smarterbalanced.org/library/en/elaliteracy-alds-and-college-content-readiness-policy.pdf

Smarter Balanced. (2013b, April 26). Initial achievement level descriptors and collect content-readiness policy (Mathematics). Retrieved from https://portal.smarterbalanced.org/library/en/mathematics-alds-and-college-content-readiness-policy.pdf

Thurlow, M. L. (2014). Common core for all—Reaching the potential for students with disabilities. Social Policy Report, 28(2), 18-20. Retrieved from https://files.eric.ed.gov/fulltext/ED566685.pdf

Thurlow, M. L., Lazarus, S. S., & Bechard, S. (Eds.). (2013). Lessons learned in federally funded projects that can improve the instruction and assessment of low performing students with disabilities. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., Lazarus, S. S., Larson, E. D., Albus, D. A., Liu, K. K., & Kwong, E. (2017). Alternate assessments for students with significant cognitive disabilities: Participation guidelines and definitions (NCEO Report 406). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., Wu, Y. C., Lazarus, S. S., & Ysseldyke, J. E. (2016). Special education—Non-special education achievement gap in math: Effects of reporting methods, analytical techniques, and reclassification. Exceptionality, 24(1), 32-44.

Towles-Reeves, E., Kearns, J., Flowers, C., Hart, L., Kerbel, A., Kleinert, H., Quenemoen, R., & Thurlow, M. (2012). Learner characteristics inventory project report (A product of the NCSC validity evaluation). Minneapolis, MN: University of Minnesota, National Center and State Collaborative.

Towles-Reeves, E., Kearns, J., Kleinert, H., & Kleinert, J. (2009). An analysis of the characteristics of students taking alternate assessments based on alternate achievement standards. Journal of Special Education, 43(4), 241-254.

Turnbull, H. R., Turnbull, A. P., & Cooper, D. H. (2018). The Supreme Court, Endrew, and the appropriate education of students with disabilities. Exceptional Children, 94(2), 124-240.

U.S. Department of Education. (2002). Twenty-fourth annual report to Congress on the implementation of the Individuals with Disabilities Education Act. Washington, DC: Author. Retrieved from https://www2.ed.gov/about/reports/annual/osep/2002/index.html

U.S. Department of Education. (2017, December 7). Questions and answers (Q&A) on U. S. Supreme Court Case Decision Endrew F. v. Douglas County School District Re-1. Retrieved from https://sites.ed.gov/idea/files/qa-endrewcase-12-07-2017.pdf

U.S. Department of Education. (2018a, January). 39th annual report to Congress on the implementation of the Individuals with Disabilities Education Act, 2017. Washington, DC: Office of Special Education and Rehabilitative Services. Retrieved from https://www2.ed.gov/about/reports/annual/osep/2017/parts-b-c/39th-arc-for-idea.pdf

U.S. Department of Education. (2018b). NAEP 2017 Mathematics and reading results. Retrieved from https://nces.ed.gov/nationsreportcard/

Valentino, A. (2006). The Individuals with Disabilities Education Improvement Act: Changing what constitutes an appropriate education. Journal of Law and Health, 20(1), 139-168.

Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnston III. Itasca, IL: Riverside Publishing.

Wu, Y.-C., & Thurlow, M. (2017). 2014-2015 APR snapshot #15: AA-AAS participation and performance. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Yell, M. L., Conroy, R., Katsiyannis, A., & Conroy, T. (2013). Individualized education programs and special education programming for students with disabilities in urban schools. Fordham Urban Law Journal, 41, 669-714.

Zirkel, P. (2017a). An adjudicative checklist of the four criteria for FAPE under the IDEA. West’s Education Law Reporter, 346, 18-20.

Zirkel, P. (2017b). Endrew F. after six months: A game changer? West’s Education Law Reporter, 348, 585-596.

Zirkel, P. (2017c). The Supreme Court’s decision in Endrew F. v. Douglas County School District RE-1: A meaningful raising of the bar? West’s Education Law Reporter, 341, 545-554.

Zirkel, P. (2018). The aftermath of Endrew F. one year later: An updated outcomes analysis. West’s Education Law Reporter, 352, 448–455.


Appendix A

Derivation of the Statement that 85%–90% of Students with Disabilities Can Meet the Same Standards as Other Students

With the 1997 reauthorization of the Individuals with Disabilities Education Act (IDEA), educators and policymakers started asking about expectations for students with disabilities. The reauthorization included for the first time a requirement for states to develop an alternate assessment for those students unable to participate in the regular assessment even with accommodations, as well as full inclusion of all students with disabilities in all state and district assessments. Then, with the reauthorization of the Elementary and Secondary Education Act (ESEA) in 2001, additional questions arose about how many students with disabilities could be expected to achieve the same content and performance standards as their peers.

Concerns were raised about the appropriateness of expecting students with disabilities to perform at the same levels as their peers without disabilities, suggesting that inability to achieve is why students receive special education services. In other words, there was a pervasive belief among educators that students with disabilities by definition cannot learn or achieve the same as their peers without disabilities. Proposals for “out-of-level” testing grew out of this belief, suggesting instead we could expect students with disabilities to perform the same as students at a lower grade level, and hold systems accountable to that assumption. Some observers noted that many students with disabilities were not being taught what other students at their grade level were being taught, instead being given a simplified or lower level curriculum, so it was unfair to the student to be tested on their grade level.

NCEO conducted research on out-of-level testing (Elliott & Thurlow, 1999; Minnema, Thurlow, Belinski, & Scott, 2001; Thurlow, Belinski, Minnema, & Scott, 2001) and worked with McGrew and Evans (2004) to explore the performance of students with low ability levels. An essential question was whether special education placement typically resulted from an inability to learn the same grade-level content as their peers and thus no more could be expected; or whether special education placement was meant to obviate any disability barriers so that students with disabilities could learn the same grade-level content as their peers, given specialized instruction, services, supports, and accommodations for that learning. Researchers found that defining what could be expected of students with disabilities by current performance was confounded by reports and concerns about their limited opportunity to learn (Nolet & McLaughlin, 2005; Shriner, Ysseldyke, Thurlow, & Honetschlager, 1994; Ysseldyke, Thurlow, & Shriner, 1992).

It was about this time that NCEO attempted to provide the field with a “bright line” to help policymakers, administrators, and educators think about how many students with disabilities could be expected, if provided appropriate instruction, supports, and accommodations, to achieve at levels comparable to their peers without disabilities. The reports and concerns about limited opportunities to learn made working from extant achievement data questionable, so a deductive approach based on what we understand about disability characteristics was used to arrive at statistics to guide the work.

To create this bright line, in 2002, NCEO examined 2000–2001 child count data. It considered the 13 disability categories and noted that there were several categories that by definition at that time did not include intellectual disabilities. Those categories that on their own (i.e., not occurring with one or more other disabilities) were not considered to include intellectual disabilities were speech or language impairments, emotional disturbance, specific learning disabilities, hearing impairments (which at that time included deafness), orthopedic impairments, other health impairments, visual impairments, deaf-blindness, and traumatic brain injury.

In school year 2000–2001, the categories of students that did not include intellectual disabilities totaled 86.5% of children eligible for special education under IDEA. The categories that could include intellectual disabilities totaled 13.4%. Further, based on an analysis of a database of IQ measures and achievement tests, McGrew and Evans (2004) found that many students with intellectual disabilities or low IQs are able to perform at grade level. They observed that it is very difficult to predict which of these students with low IQs will learn to grade level when given the opportunity to learn grade-level content, but unless they are expected to learn they will not achieve at that level.

In addition, NCEO looked at estimates based on states that broke down the intellectual disabilities category into mild, moderate, and severe, and found that the percentage of students with mild to moderate intellectual disabilities generally constituted the majority of students in this category. To acknowledge that some students with low IQs and the label of intellectual disabilities could learn to grade level when taught well, we estimated conservatively that a small percentage of students in that category should be expected to learn to grade level. That is, NCEO made a bright line estimate of 10–15% of students possibly not being able to achieve grade-level performance, even if provided the best instruction, supports, and accommodations, due to intellectual disabilities. In other words, 85% to 90% of all students with disabilities can be expected to achieve grade-level achievement when provided with the best instruction, supports, and accommodations to go around the barriers of their disabilities to the grade-level content expected for all students. This information was provided by Thurlow to congressional hearings in 2004 and 2010.

Once all students with disabilities are receiving the best instruction, supports, and accommodations to learn the grade-level general curriculum, we will be able to determine how close this estimate is.

References

Elliott, J. L., & Thurlow, M. L. (1999). Out-of-level testing. The School Administrator, 56(10), 18–23.

McGrew, K. S., & Evans, J. (2004). Expectations for students with cognitive disabilities: Is the cup half empty or half full? Can the cup flow over? (Synthesis Report 55). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Minnema, J. E., Thurlow, M. L., Bielinski, J., & Scott, J. K. (2001). Past and current research on out-of-level testing of students with disabilities. Assessment for Effective Intervention, 26(2), 49–55.

Nolet, V., & McLaughlin, M. J. (2005). Assessing the general curriculum: Including students with disabilities in standards-based reform. Thousand Oaks, CA: Corwin Press.

Shriner, J. G., Ysseldyke, J. E., Thurlow, M. L., & Honetschlager, D. (1994). “All” means “all”: Including students with disabilities in performance-based systems. Educational Leadership, 51(6), 38–42.

Thurlow, M. L. (2004, March 3). Testimony of Martha L. Thurlow for the hearing on NCLB and the inclusion of students with disabilities in the accountability and assessment systems. Available at https://rtc3.umn.edu/docs/OnlinePubs/EdWorkforceTestimony.pdf

Thurlow, M. L. (2010, April 28). Written testimony of Martha L. Thurlow for the hearing on ESEA reauthorization: Standards and assessments. Available at https://rtc3.umn.edu/docs/Presentations/ThurlowTestimony2010.pdf

Thurlow, M., Bielinski, J., Minnema, J., & Scott, J. (2002). Out-of-level testing revisited—A new issue for a new time. In G. Tindal & B. Haladyna (Eds.), Large-scale assessment programs for all students: Development, implementation and analysis (pp. 453–466). Mahwah, NY: Lawrence Erlbaum.

Ysseldyke, J. E., Thurlow, M. L., & Shriner, J. G. (1992). Outcomes are for special educators too. Teaching Exceptional Children, 25(1), 36–50.


Appendix B

Performance Level Descriptors of DLM Alternate Assessments in Grade 4 English Language Arts Alternate Assessments Year-End Model

Emerging: A student who achieves at the emerging performance level typically can identify familiar people, objects, or places; identify text elements; and demonstrate an understanding of language when reading literature and informational text.

The student identifies people, objects, or places associated with a text by

  • Attending longer to a new object that has been added to a pair of familial identical object
  • Indicating a similar object from a group of two similar objects and one different object
  • Indicating a different object from a group of two identical objects and one different object
  • Naming objects from pictures
  • Indicating objects or pictures from named categories
  • Indicating familiar people, objects, or places

The student identifies text elements by

  • Identifying concrete details in a familiar story
  • Identifying character actions
  • Identifying major events

The student demonstrates an understanding of language by

  • Identifying words with similar or different meanings
  • When writing, the student
  • Identifies familiar people, objects, or places
  • Understands object names
  • Understands that letters are used to write words
  • Identifies the first letter of his or her name
  • Recognizes when a letter is uppercase or lowercase

Approaching the Target: A student who achieves at the approaching the target performance level typically can identify text elements, demonstrate an understanding of the language, and identify text structure when reading literature and informational text.

The student identifies text elements by

  • Describing characters
  • Identifying how characters’ actions result in consequences
  • Identifying the main points
  • Identifying the theme of a familiar story

The student demonstrates an understanding of language by

  • Selecting appropriate words to complete literal sentences
  • Providing real-world connections between words and their uses

The student identifies text structure by

  • Using pictures or objects related to the text to learn additional information
  • Identifying the beginning, middle, and end of a text
  • Determining when two different texts on the same topic make a similar statement

When writing, the student

  • Identifies words that describe familiar people, objects, or places
  • Uses letters to create words
  • Demonstrates an understanding of capitalization

At Target: A student who achieves at the at target performance level typically can identify text elements, demonstrate an understanding of language, and identify text structure when reading literature and informational text.

The student identifies text elements by

  • Identifying details related to people, events, or ideas
  • Comparing key details
  • Identifying reasons that support points in a text
  • Identifying the topic of a text
  • Identifying the theme of a familiar story

The student demonstrates an understanding of language by

  • Determining the meaning of unambiguous words in a text
  • Identifying words with opposite meanings
  • Determining which words in a text relate to explicit information

The student identifies text structure by

  • Identifying the characteristics of a text
  • Determining when two different texts on the same topic make a similar statement

When writing, the student

  • Identifies words, facts, details, or other information related to a topic
  • Spells words phonetically using letter-sound knowledge and common spelling patterns
  • Capitalizes the first letter of a sentence

Advanced: A student who achieves at the advanced performance level typically can identify text elements, demonstrate an understanding of language, and identify text structure when reading literature and informational text

The student identifies text elements by

  • Using details to describe characters, settings, and events
  • Using details to answer questions
  • Identifying relationships between specific points and the reasons supporting them
  • Identifying the overall topic of a text

The student demonstrates an understanding of language by

  • Determining which words in a text relate to the topic
  • Identifying words with similar meanings
  • Identifying the meaning of words with multiple meanings

The student identifies text structure by

  • Identifying elements of a story that change from the beginning to the end
  • Determining if a text provides information about events, gives directions, or provides information on a topic
  • Comparing and contrasting details in two texts

When writing, the student

  • Provides facts, details, or other information related to the topic
  • Spells words with inflectional endings
  • Uses correct capitalization when writing a title

Appendix C

Performance Level Descriptors of MSAA Grade 4 Mathematics Alternate Assessments

Figure 4 Table