States’ Alternate Assessments Based on Modified Achievement Standards (AA-MAS) in 2007 Synthesis Report 67 Sheryl S. Lazarus, Martha L. Thurlow, Laurene L. Christensen, & Damien Cormier December 2007 All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Lazarus, S. S., Thurlow, M. L., Christensen, L. L., & Cormier, D. (2007). States’ alternate assessments based on modified achievement standards (AA-MAS) in 2007 (Synthesis Report 67). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Table of Contents Process Used to Find Information About States’ AA-MAS Appendix A: State Documents Used in Analysis Appendix B: AA-MAS Characteristics by State
In April 2007, No Child Left Behind regulations were finalized that gave states the option to develop an alternate assessment based on modified achievement standards (AA-MAS). This assessment option is for a small group of students with disabilities who can make significant progress, but who may not reach grade-level achievement within the time period covered by their Individualized Education Program (IEP) (U.S. Dept. of Education, 2007). Prior to the finalization of this regulation a few states had developed, or were developing, an assessment they considered to be an AA-MAS—though none had yet been through the U.S. Department of Education’s peer review process. This study compiles and summarizes publicly available information about these assessments. In July 2007 five states—Kansas, Louisiana, North Carolina, North Dakota, and Oklahoma—had an assessment they considered to be an AA-MAS. In addition to these states, Maryland had information on its state Web site about an AA-MAS that was under development and designated to be implemented during the 2008-09 school year at the earliest. Most states with this assessment option used a multiple-choice test; sometimes, there was also a writing prompt or items that required a constructed response. One state had a portfolio assessment. The eligibility criteria for the AA-MAS differed across states, but all states required the student to have an IEP. Other criteria that many states included were: decision not based on categorical label; student does not have significant cognitive disabilities; student performance level is not due to excessive absences or to social, cultural, environmental, or economic factors; and student is learning grade-level content. States’ AA-MAS’s differed in a number of ways from their regular assessments. For the AA-MAS, some states removed a distractor, had fewer items, had shorter passages, or used simplified language. States often incorporated some accommodations into the design of the AA-MAS. The ones mostly frequently incorporated were larger font size and fewer items per page. Federal legislation requires that all students, including students with disabilities, be included in all state and district-level accountability systems. Many students can take the regular assessment with or without accommodations, but some students with disabilities need alternate ways to access assessments. For the past several years, states have had alternate assessments based on alternate achievement standards. In April 2007 No Child Left Behind (NCLB) regulations on modified achievement standards were finalized. These regulations were designed to give states additional flexibility. As described in the regulations (U.S. Department of Education, 2007), states have the option of providing an alternate assessment based on modified achievement standards (AA-MAS). Students who participate in this assessment may be from any disability category. Their progress to date, in response to appropriate instruction, must be such that the student is unlikely to achieve grade-level proficiency within the year covered by the IEP. The regulations require that students who participate in this option have access to grade-level content. Several states had an alternate assessment based on modified achievement standards in place, or in development, prior to the regulations that they believe meet the criteria of an AA-MAS—even though none have yet been through the peer review process used by the U.S. Department of Education to determine whether the assessment fulfills the necessary requirements for the state to receive Federal funds. The purpose of this report is to provide a snapshot of the characteristics of the AA-MAS in these states at a time shortly after the April 2007 regulations were finalized. Because these states developed their assessments prior to the final regulations, some of the characteristics of these early AA-MAS may not fully comply with the regulations. We did not attempt to determine the degree to which these assessments meet the Federal requirements. Those determinations will be made through the official peer review process that requires states to demonstrate that their assessment systems used for accountability purposes meet certain criteria. To prepare for this peer review process, states compile a set of relevant materials and evidence (e.g., state statutes and regulations, test administrator manuals, assessment reports, etc.). The peer reviewers examine this evidence under the guidance of a U.S. Department of Education staff member. The Assistant Secretary for Elementary and Secondary Education uses the peer reviewers’ comments to make decisions about approval of a state’s assessment system. The landscape of AA-MAS development is changing rapidly, and we anticipate that additional states may have an AA-MAS soon. It is possible that some other states have a version of an AA-MAS that is not yet public. Some states may be considering the development of an AA-MAS; others may be field testing items. This study addressed only those AA-MAS for which public information was available in July 2007. Specific questions that we sought to answer in this study included: 1. In July 2007 which states either had an assessment that they considered to be AA-MAS or had information about an AA-MAS in development on their Web site? 2. What were the characteristics of these assessments? 3. What were the eligibility criteria for students to qualify to participate in this assessment option? Process Used to Find Information About States’ AA-MAS This report summarizes publicly available information about the characteristics of the AA-MAS for states that either had one in place in July 2007 or had information about an AA-MAS in development on the state Web site in July 2007. Data were gathered from state Web sites as well as from presentations at the Council of Chief State School Officers’ (CCSSO) Large-Scale Assessment Conference in Nashville, Tennessee in June 2007. Several states made presentations at the conference on their AA-MAS and their PowerPoint slide handouts were used as a data source. Appendix A contains a list of the documents used to compile the information in this report. AA-MAS information was collected for each state and placed into a state profile. The profiles were then e-mailed to each state in September 2007. States were asked to verify the information; if the profile contained inaccurate information, states were permitted to revise their profiles. We then compiled and summarized the verified information in this report. As indicated in Table 1, five states—Kansas, Louisiana, North Carolina, North Dakota, and Oklahoma—had an assessment that they considered to be an AA-MAS. In addition to these states, Maryland had information on its Web site about an AA-MAS under development. Some states had developed their AA-MAS for more grades and content areas than others. For example, grade 4 was the earliest grade at which the LAA 2 (LEAP Alternate Assessment, Level 2) in Louisiana was available; the other five states had an AA-MAS for at least some content areas, starting in Grade 3. The six states that had, or are developing, an assessment that they believed to be an AA-MAS, differ from one another. As shown in Figure 1, one state—North Dakota—had a portfolio assessment, while the other five states had a multiple-choice assessment. One state’s assessment also included some constructed responses; two states had a writing prompt. Additional details about the types of assessments are available in Appendix B in Table B-1. Table 1. AA-MAS Name, Content Areas, and Grade Described by State
1 Still under development. Maryland planned to implement its AA-MAS in 2008-09 at the earliest. Figure 1. Number of States with Selected Assessment Types and Question Characteristics
States with an assessment they considered to be an AA-MAS had eligibility criteria that IEP teams use to determine which students qualify to participate in this assessment option. Some states had developed decision trees, while others had checklists or descriptions of the eligibility criteria. Table 2 shows that all six states in this study required that a student have an IEP to be a candidate for this assessment option. Other frequently listed criteria included: student multiple years behind grade level expectations (n=4); not based on student’s categorical label (n=4); student does not have significant cognitive disabilities (n=4; and not due to student’s excessive absences or to social, cultural, environment, or economic factors (n=4). Table 2. Number of States with Selected Eligibility Criteria 1
1 In addition to the eligibility criteria listed in this table, 5 states have other criteria. See Table B-2a for details about these other criteria.Half of the states (n=3) required that the student must be learning grade-level content, while fewer states required that a student had previously scored at an unsatisfactory level on the state assessment (n=2) or that the decision not be based on student’s placement setting (n=2). Because this group of states had an assessment in place or in development prior to the April 2007 regulation, some states’ criteria may have differed from regulatory requirements. Additional details about the states’ eligibility criteria are available in Appendix B in Tables B-2 and B-2a. Design Changes The assessments that states consider to be an AA-MAS differed in a number of ways from the states’ regular assessments. The states made a number of design changes for the AA-MAS. As indicated in Figure 2, four states removed a distractor. For example, if multiple-choice questions on a state’s regular assessment had four possible answer options, then the AA-MAS would have three possibilities. Most states also simplified the language (n=4) and had fewer items (n=4) on the AA-MAS than on the state’s regular assessment. Three states had shorter reading passages, while one state segmented reading passages so that questions directly followed the section of the passage to which they referred. Examples of other design changes included shorter responses to the writing prompt and the requirement that all of the items be Depth of Knowledge (DOK) levels 1 or 2. The latter requirement refers to methodologies that can be used to organize the level of understanding required for various topics or items. The higher the level, the deeper the level of understanding needed. Table B-3 in Appendix B contains state specific information as well as details about the other changes. Figure 2. Selected Design Changes in States’ AA-MAS
Accommodations Some states had incorporated features that would be considered an accommodation on the state’s regular assessment into the design of the assessment considered by the state to be an AA-MAS (see Figure 3). Several states had fewer items per page (n=3). In some states the regular math assessment had one or more sections where the calculator may not be used except if a student has that accommodation listed on the IEP. Two such states incorporated the use of a calculator into the AA-MAS assessment design. Other accommodations incorporated into the assessment design included larger font size (n=3), breaks as needed (n=2), and key text underlined or bolded (n=1). More detailed state specific information about accommodations incorporated into the design of these assessments is presented in Appendix B, Table B-4. Figure 3. Accommodations Incorporated into AA-MAS
In July 2007 six states had an assessment either in place or in development that they considered to be an AA-MAS, but none had as of yet gone through the U.S. Department of Education’s peer review process. This study compiled and summarized information about these assessments. Key findings included: For its AA-MAS, most states had multiple-choice test formats. Sometimes, there was also a writing prompt or items that required a constructed response. One state had a portfolio assessment. The eligibility criteria for the AA-MAS differed across states, but in all states students were required to have an IEP to qualify for this assessment option. Other criteria that many states had included the following: decision cannot be based on a categorical label; student does not have significant cognitive disabilities; student’s low performance is not due to excessive absences or to social, cultural, environmental, or economic factors; and student is learning grade-level content. States’ AA-MASs differed in a number of ways from their regular assessments. For the AA-MAS, some states removed a distractor, had fewer items on the AA-MAS, had shorter passages, or used simplified language. States often incorporated some accommodations into the design of their AA-MASs. The ones mostly frequently incorporated were larger font size and fewer items per page. Now that the regulations for alternate assessments based on modified achievement standards are final, more states are considering the development of an AA-MAS. The population of students for whom this policy applies is a small group of students with disabilities who can make significant progress, but who may not reach grade-level achievement within the time frame covered by their IEP (U.S. Dept. of Education, 2007). Important considerations for states that are exploring this option include consideration of how to identify who the students are that might participate in this option, how the students access grade-level content, and how the students show what they know (National Center on Educational Outcomes, 2007). According to Marion (2007), states need to be "prepared to build a defensible validity argument in support of this assessment and in particular should be prepared to address validity issues related to potential unintended negative consequences as a result of implementing this assessment (e.g., lower expectations)" (p. 5). For example, states need to consider, "How will the inclusion of the AA-MAS as part of the state’s assessment system lead to better instructional and curricular opportunities for these students? Whatever benefit (e.g., positive consequential evidence) for these students should be weighed against potential benefits from other approaches such as more appropriate accommodations and especially targeted instructional interventions" (p. 5). In this report we make no evaluative comments about the various approaches to AA-MAS that states have taken. For information about relevant research, refer to the NCEO Web. This report provides information about the characteristics of some of the first state assessments that may be an AA-MAS, but we anticipate that this report is merely the first snapshot of a rapidly changing landscape since it is anticipated that more states will develop an AA-MAS. Marion, S. (2007, July 26). A technical design and documentation workbook for assessments based on modified achievement standards. Minneapolis MN: University of Minnesota, National Center on Educational Outcomes. Retrieved September 12, 2007 from the World Wide Web: http://cehd.umn.edu/NCEO/Teleconferences/AAMASteleconferences/AAMASworkbook.pdf National Center on Educational Outcomes (2007). Identifying students with disabilities who are eligible to take an alternate assessment based on modified academic achievement standards. Minneapolis, MN: University of Minnesota, Author. Retrieved September 12, 2007 from the World Wide Web: http://cehd.umn.edu/NCEO/Teleconferences/AAMASteleconferences/AAMASIdentifying%20Students.pdf U.S. Department of Education (2007, April 9). Final Rule 34 CFR Parts 200 and 300: Title I—Improving the Academic Achievement of the Disadvantaged; Individuals with Disabilities Education Act (IDEA). Federal Register. 72(67), Washington DC: Author. Retrieved September 12, 2007 from the World Wide Web: http://cehd.umn.edu/NCEO/2percentReg/FederalRegApril9TwoPercent.pdf Appendix A: State Documents Used in Analysis State documents and presentations used in the analysis of states’ alternate assessments based on modified achievement standards.
Appendix B: AA-MAS Characteristics by State Table B-1. Assessment Type and Question Characteristics
1 Still under development. Maryland planned to implement an AA-MAS in 2008-09 at the earliest. In addition, specific accommodations implemented in these assessment/instructional settings may include: test items are less complex, fewer and shorter passages, shorter or less difficult questions, and fewer distractors.
Table B-2. Eligibility Criteria
1 Still under development. Maryland planned to implement an AA-MAS in 2008-09 at the earliest.2 KS: Consistently requires instruction in prerequisite skills to the grade level indicators being assessed; despite the provision of research-based interventions, the student is not progressing at the rate expected for grade level; classroom achievement and performance is significantly below grade level peers.3 LA: Must be functioning at least 3 grade levels below actual grade in English language arts and/or mathematics.4 ND: Differentiates between significant and persistent cognitive disabilities. Students with persistent cognitive disabilities would participate in the AA-MAS (e.g., students with significant cognitive disabilities would participate in the AA-AAS).5 MD: Requires use of a modified general curriculum that is aligned with Maryland Content Standards for the student’s grade level, but is modified (reduced amount to learn, reduced complexity, reduced output); requires and receives modifications in instruction (examples include: reduced complexity of language, paraphrasing of reading passages, reduced number of test items, reduced amount of content to learn, embedded scaffolding for a written response such as sentence stems, guided response outline, guided questioning to generate response, software such as Co-Writer and Write Outloud, use of a calculator, and spell check).6 LA: Not available in Grade 3 because a participation criterion is that the student must score at the unsatisfactory level on the general assessment.
Table B-2a. Other Eligibility Criteria Specifications
1 Still under development. Maryland planned to implement an AA-MAS in 2008-09 at the earliest.
Table B-3. Comparison of AA-MAS and Regular Assessment: Design Changes
1 Only at some grade levels (e.g., upper grades).2 Still under development. Maryland planned to implement an AA-MAS in 2008-09 at the earliest.3 North Dakota has a portfolio assessment.
Table B-4. AA-MAS Accommodations and Comparison of Accommodations with Regular Assessment, Selected States
1 Still under development. Maryland planned to implement an AA-MAS in 2008-09 at the earliest. |