Synthesis Report 88Alternate Assessments Based on Alternate Achievement Standards (AA-AAS) Participation PoliciesDebra Albus & Martha L. Thurlow December 2012 All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Albus, D., & Thurlow, M. L. (2012). Alternate assessments based on alternate achievement standards (AA-AAS) participation policies (Synthesis Report 88). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Table of Contents
Executive SummaryThe participation of all students in state assessments used for accountability has been an increasingly emphasized policy and practice push since the mid-1990s. In recent years, there has been a shift in emphasis to consortia of states working together to develop new assessment systems based on common standards that stress college and career readiness. Most states have adopted a set of English Language Arts and Mathematics standards developed by the National Governors' Association and the Council of Chief State School Officers, along with a host of other organizations (see www.corestandards.org). Four consortia have been funded to develop new assessment systems: two that are developing regular assessment systems (Partnership for Assessment of Readiness for College and Careers--PARCC, and Smarter Balanced Assessment Consortium--SBAC), and two that are developing systems for alternate assessments based on alternate achievement standards (Dynamic Learning Maps--DLM, and National Center and State Collaborative--NCSC). Since 1992, the National Center on Educational Outcomes (NCEO) has analyzed participation and accommodation policies for students with disabilities on state administered assessments, and has developed reports on policies for regular assessments, alternate assessments based on modified achievement standards (AA-MAS), and alternate assessments based on alternate achievement standards (AA-AAS). This report focuses on participation policies for AA-AAS. Publicly available participation guidelines were compiled and analyzed for 58 states and entities (Washington DC, Guam, etc.). In addition to providing a national picture of policies, we have organized the results of our policy analysis by each of the four content assessment consortia (PARCC, SBAC, DLM, NCSC). Participation Options Across StatesStates and consortia had different policies about whether to allow combined participation (i.e., student may take the AA-AAS for one content area and an AA-MAS or regular assessment for another). Eleven states and entities allowed combined participation, while four states did not allow it. Three states did not allow selective participation in which a student participated in the AA-AAS for one or more content areas, but was not assessed in other content areas. For the other participation options, seven states allowed participation adjusted for testing window and three states allowed participation adjusted for disability. Depending on the consortia, one to six states allowed combined participation across consortia, and one to two states did not. Across three of the consortia, two states also allowed adjusted participation based on a student's disability, by allowing a different form of the alternate or by allowing certain items to be exempt for certain students. Consortia working on developing an AA-AAS should address whether their policies will allow for combined participation; this decision is one that is likely to have an impact on the regular assessment consortia as well as the AA-AAS consortia. One to four states in the consortia allowed adjustments in participation based on student arrival or decision timing in relation to the AA-AAS testing window. Some states in our review limited the participation of late arriving students by reducing the number of items completed within a content area, by reducing the number of content areas attempted, or by allowing a student's file to be submitted without sample work, with or without a petition for exemption based on time of arrival. It would be useful for states to anticipate in policy what flexibility will be allowed, and what logistical expectations, specific to transfer of student materials, may be set for different types of transfers (e.g., within and outside a consortium) in relation to testing windows. Criteria Allowed and Not Allowed Across StatesAll states' written participation policies for the AA-AAS addressed the criteria that could and could not be used for making participation decisions. Criteria mentioned by at least five states were numerous, including 25 criteria that could be used for AA-AAS participation decisions and 14 criteria that could not be used. The top two criteria listed by states were that the student has a significant cognitive disability, and that the decision is made by the IEP team; both of these were included in their written policies by 49 states and entities. The most frequently listed criterion that could not be used was disability label or characteristics (n=40). A small number of states and entities either required IQ scores or mentioned a typical score range as a suggestion for participation decisions. It is important for states to discuss when criteria are requirements and when they are simply suggestions. It is not clear how differences across states in their participation criteria might affect the goal of ensuring that the appropriate population of students participates in the AA-AAS. Training and professional development will benefit from consistent materials within a consortium. Several factors will enter into this:
Decisions for English Language Learners (ELLs)Several states mentioned English Language Learner (ELL) status of students and how this might affect decisions about participation in the AA-AAS. Most often, the policies mentioned an allowance for new arrivals to the country being exempt from participation in Reading/ELA assessments. An important factor to consider in decision-making is to ensure that eligibility criteria are communicated within a state in a consistent way for all population groups across documents. Several states gave specific instructions to be used for ELLs either in the same document as for all students with IEPs or in a separate ELL document. For states that choose to present this information in multiple documents, it is important to keep content updated in both the special education and ELL documents, or to note on older documents that certain information is outdated. States' references to ELLs in AA-AAS participation decision rarely distinguished between ELL status and English proficiency, and the implications of those differences for making decisions about participation in an AA-AAS. Greater clarity about the participation of ELLs with significant cognitive disabilities probably is needed. Top of Page | Table of Contents OverviewThe participation of all students in state assessments used for accountability has been an increasingly emphasized policy and practice push since the mid-1990s (Shriner, Ysseldyke, Thurlow, & Honetschlager, 1994; Thurlow & Ysseldyke, 1997). Students with the most significant cognitive disabilities participate in alternate assessments based on alternate achievement standards (AA-AAS). Most students with disabilities participate in the regular assessment, with or without accommodations. A few states also have an alternate assessment based on modified achievement standards (AA-MAS) or alternate assessment based on grade-level achievement standards (AA-GLAS) that some students with disabilities take. In recent years, there has been a shift in emphasis to consortia of states working together to develop new assessment systems based on common standards that stress college and career readiness (Achieve, 2010; Porter, McMaken, Hwang, & Yang, 2011). Most states have adopted a set of English Language Arts and Mathematics standards developed by the National Governors' Association and the Council of Chief State School Officers, along with a host of other organizations (see www.corestandards.org). Four consortia have been funded to develop new assessment systems: two that are developing regular assessment systems (Partnership for Assessment of Readiness for College and Careers--PARCC, and Smarter Balanced Assessment Consortium--SBAC), and two that are developing systems for alternate assessments based on alternate achievement standards (Dynamic Learning Maps--DLM, and National Center and State Collaborative--NCSC). States have developed participation guidelines, containing a set of criteria, for determining which students are eligible to participate in the different assessment options. Individualized Education Program (IEP) teams use participation guidelines to determine which assessment option is appropriate for individual students. Currently different states in the each consortium have different criteria for participation in their AA-AAS. States in the consortia have different perspectives on how students with disabilities are included in their assessments, and the consortia need to develop shared participation criteria (NCEO, 2011). Since 1992, the National Center on Educational Outcomes (NCEO) has analyzed participation and accommodation policies for students with disabilities on state administered assessments, and these generally have focused almost entirely on policies for regular assessments (see Christensen, Braam, Scullin, & Thurlow, 2011), with other assessments, such as the alternate assessment based on modified achievement standards (AA-MAS), and alternate assessments based on achievement standards (AA-AAS) merged in with the policies for the regular assessment. Separate reports were developed for the AA-MAS (Lazarus, Hodgson, Price, & Thurlow, 2011; Lazarus, Hodgson, & Thurlow, 2010; Lazarus, Rogers, Cormier, & Thurlow, 2008). This report focuses on participation policies for the AA-AAS. In addition to providing a national picture of policies, we have organized the results of our policy analysis by each of the four content assessment consortia (PARCC, SBAC, DLM, NCSC). Prior to this report, the most recent study of participation guidelines focused solely on the AA-AAS (Musson, Thomas, Towles-Reeves, & Kearns, 2010) examined policies for the 50 states that were available during a window of time in October, 2007. That study included state documents from 2003-04 to 2007-08. This report provides an updated analysis of state participation policies for the AA-AAS. It examines information on both the participation options that states provide and the criteria that states indicate may or may not be used in making decisions about participation in the AA-AAS. Top of Page | Table of Contents MethodsNCEO staff reviewed state and special entity department of education websites, and collected information on participation policies in August and September of 2011. The data were then compiled and analyzed. (See Appendix A for a list of the documents used in this analysis.) Staff compiled notes using Excel on participation options. The notes categorized the participation criteria using a list of explicit criteria that states allowed or did not allow in making decisions about participation in AA-AAS. Additional information also was collected on participation options and adjustments to participation (see Results for definitions). Individualized profiles were then developed for each state that contained the participation information that had been compiled. The profiles were then verified with the states to help ensure that the most accurate and current data had been compiled. An e-mail explaining our request for verification, with the individualized state form, was sent to state assessment directors or to state AA-AAS staff, if they were noted on the state website. These e-mails were sent on November, 21, 2011 and the requested deadline for response was December 23, 2011. A short extension to this deadline was allowed for gathering and clarifying data into early January, 2012. If a state requested changes, we required evidence of a written document before accepting the change. Overall, 36 states and entities verified data in response to the request. As a result of feedback during verification, NCEO refined the analysis categories to better reflect how states were using terms. This approach to categorizing state data resulted in fewer assumptions about uses of terms. We obtained policy data from the 50 states, the District of Columbia, and seven special entities (American Samoa, Federated States of Micronesia, Guam, Republic of the Marshall Islands, Republic of Palau, Commonwealth of Northern Mariana Islands, and Virgin Islands). We did find general information online about the AA-AAS for Puerto Rico, but it did not include information about AA-AAS criteria or participation options, so is not included here. We did not gather information for the Bureau of Indian Education, which uses the policies of the states in which its schools are located. Fifty-eight states had publicly available documents that contained participation information. Several states and entities referred NCEO to additional documents during the verification process. At least one state referred to documents across more than one year of publication. In these cases, we used the date of the document or website with the most recent information to note the date of publication. Overall, 9% of the states and entities had documents from before the 2010-2011 school year, 43% had documents for the 2010-2011 school year, 41% had documents for 2011-2012 school year, and 7% of the states and entities had no date on their documents. Of the documents found for these analyses, 15% of state participation information came from regular assessment policy documents that included assessments given to all students. Other information on participation came from separate special education documents for 53% of the states and entities. Participation information that mentioned English language learners was found in 12% of states' and entities' documents. We organized the presented results both overall and for each of the four content assessment consortia. These four consortia are referred to in figures by their abbreviations: DLM: Dynamic Learning Maps Alternate Assessment Consortium, AA-AAS The state membership in the consortia was accurate as of January, 2012. Some states were in both consortia at that time, and others were in none of the consortia (Minnesota, Nebraska, and Texas). All figures in the results section of this report reflect criteria or categories that were mentioned by at least five states. More complete information, including results for criteria included in policies by fewer than five states, can be found in tables presented in Appendix B (see Table B4 and Table B6). Also, the specific policy language for criteria allowed and not allowed for each state is provided in Appendix B tables B8 and B9. Top of Page | Table of Contents ResultsResults are presented in three sections. The first section summarizes findings about participation options that were allowed or not allowed for AA-AAS. The second section addresses the specific criteria that were allowed or not allowed for making participation decisions. The third section addresses ELL status and participation criteria. Participation OptionsThis section summarizes the AA-AAS participation options found in state policy documents across all 58 states and entities that had detailed participation policy information. It presents information on the number of states that allowed or did not allow each option. Circumstances under which policies indicated that a student was not counted as participating in the AA-AAS were similar to those of the regular assessment. For example, they included student absence on day of testing, parent exemption, and student refusal. Our analysis focused on participation options specific to the AA-AAS. The options for participation would most typically be selected prior to the opening of a state's assessment window. The options are:
Figure 1 shows the participation options with the total number of states and entities that allowed or did not allow each option. Not all states and entities specifically addressed these options in their documents. Information about the various participation options were found in only 20 states' policies. As seen in Figure 1, 11 states allowed combined participation while 4 states did not allow it. Three states did not allow selective participation, in which a student participated in the AA-AAS for one or more content areas, but was not assessed in other content areas. For the other participation options, 7 states allowed adjusted participation for testing window and 3 states allowed adjusted participation for disability. Six states had other information about participation options. (See Tables B1 and B2 in Appendix B for additional details about participation options.) Figure 1. Number of States and Entities that Allow and Do Not Allow Each Participation Options* (N=20)
*N=20 states and entities with publicly available participation option information Participation Options by ConsortiaFigure 2 shows the number of states and entities in each consortium that allowed or did not allow each of the participation options. The number of states identified for each consortium (and on which the percentage is based) is the number of states and entities in the consortium; it does not indicate the number of states with information on the participation options. As shown in the figure, all consortia had at least one state that allowed and at least one state that did not allow the combined participation option. In percentages, the range was from 8% (DLM) to 25% (NCSC) that allowed combined participation and from 4% (NCSC) to 8% (DLM and PARCC) that did not allow combined participation. In three of the consortia, at least five states allow combined participation (20% to 25% of the states in the consortium). No state allowed selective participation, but one state each in DLM and NCSC, and two states each in PARCC and SBAC specifically did not allow it. Across the consortia only one to four states with available data allowed for adjusted participation for testing window. Similarly, only a few states with available data allowed adjusted participation for disability (0 in DLM to 4 in NCSC). Figure 2. Number of States and Entities by Consortia that Allow or Do Not Allow Participation Options*
*N=20 states with information, Consortia Ns are total states within each consortia Criteria for ParticipationThe following sections address participation criteria that were allowed and not allowed. Results are presented first for all states and entities, and then organized by the four consortia. The tables are structured to address criteria mentioned most frequently by states overall. This information is followed by a section on policy information for English language learners. Criteria for Participation: AllowedFigure 3 shows the 25 criteria that could be used for AA-AAS participation decisions; as described in the Methods section, this figure includes only those criteria mentioned by at least five states. The top two criteria (student has a significant cognitive disability; participation decision to be made by IEP team) were listed by 49 states and entities. The next most frequently listed criterion was the need for substantial adjustments to the student's curriculum (n = 44). Six additional criteria were listed by at least 30 states and entities (must consider eligibility criteria, has current IEP, significant supports needed to access curriculum, modifications on regular assessment compromise validity, documented reason why AA-AAS is appropriate, disability category/characteristics). Additional details about the participation criteria that are allowed are in Table B3 in Appendix B. Criteria that were listed by fewer than five states overall are listed in Appendix Table B4. For example, four states required that IQ scores be used in the decision-making process. Detailed specification notes for all the participation criteria are in Table B5. Figure 3. Number of State and Entities that Allow Specific Participation Criteria to be Used for Participation Decisions*
*N=58 Definitions of criteria: Has significant cognitive disability: This term varied by state, but was noted if language approximated the term (e.g., significant cognitive disability, significant or severe disability, or any specific criteria to define these). IEP team decision: IEP team makes decision, but explicit team members mentioned varied across states. Substantial adjustments to curriculum: Substantive changes made to curriculum of student. Has current IEP: Explicit or inferred language about IEP being current. Must consider eligibility criteria: Language that required that criteria be considered in making a participation decision, with a range from requiring that all criteria be met to a simple requirement that some or all criteria be considered; Decision-tree formats were one indication of the need to consider criteria. Significant supports needed to access curriculum: Significant supports which may include teacher supports or significant accommodations or modifications for student to access the curriculum. Modifications needed in other assessment(s) compromise validity: Modifications that student would need to participate in a regular assessment would compromise validity. This includes not being able to participate in regular assessment with either accommodations or modifications. Documented reason for why AA-AAS is appropriate: State requires documented reason for why it is appropriate for student to take AA-AAS. This does not necessarily include evidence for the reason given, but may include signed statements on forms, signed checklists, etc. Disability category/characteristics: Student's disability category (e.g., Multiple Disabilities, Traumatic Brain Injury, etc.) as well as any descriptive characteristics of challenges a student may have. Requires extensive support in daily activities: This includes extensive supports needed for daily activities with emphasis on either academic or non-academic activities during the student's day. Adaptive behavior deficit: Student has challenges in behavior needed for daily skills. (Focus here is on student adaptive skills, in contrast to supports needed for daily activities, or a curriculum component.) Curriculum may include functional and living skills: Functional or living skills are part of student's curriculum. Performance measured against alternate achievement standards: Performance either in daily assessments or assessments are measured against a state's alternate achievement standards. Individual student needs: Explicit mention of considering individual student's needs. Enrolled grade: Enrolled grade of student is used for determining aspects of participation, such as which content test to administer, or with regard to appropriate alternate achievement standards. Decision timing: Timing of a decision may refer to when a participation decision is made for a student relative to an assessment window, or the development or updating of an IEP in order to participate, etc. Current and historical data used: Past or current data to be used in making a decision; specific data required varies across states. Evidence in IEP for each criteria required: State requires evidence for each criterion in IEP to participate. This may include evidence for sub-criteria. Ensure instruction is linked to standards: Requires that instruction student had or currently has is linked to the state's standards. Informed parents about consequences: Parents are informed about state or district consequences of a student's participation in the AA-AAS. Communication deficits: Receptive or expressive difficulties in communication. (This may overlap with communication mode, but is separate here to reflect broad policy that does not specify a mode.) Very low achievement compared to same aged peers: Student achievement in any content area is very low compared to other students of the same age. Not expected to reach regular diploma: Student is not expected to complete a standard academic curricula that leads to a regular diploma. Uses devices/assistive technology: General or specified uses of various devices or assistive technology to provide access to program of study. Communication mode: Receptive or expressive difficulties in communication that specifies a mode (e.g., eye gaze, cries, facial expressions, regularized gestures, signs, pictures, etc.) The exact specifications of criteria differed across states. For example, the criterion has a significant cognitive disability, was explained by states in different ways. Examples of state specifications are: Arizona: Evidence of a Significant Cognitive Disability: Empirical evidence (formal testing results, multidisciplinary evaluation team results, etc.) of a significant cognitive disability prevents the acquisition of the grade‐level Arizona Academic Content Standards. Please note that students with learning disabilities who have overall intellectual and/or adaptive behavior abilities within the average range are not students with most significant cognitive disabilities. The student functions like a student with MR across all areas: commensurate abilities in mathematics, reading, and writing, adaptive behavior scores, and measures of intellectual abilities. Colorado: Does the student have an Individualized Education Program (IEP)? AND Evidence of a Significant Cognitive Disability? Georgia: Has significant cognitive disability: Does the student have significant intellectual disabilities or a combination of intellectual disabilities with motor, sensory, or emotional behavior disabilities? Nebraska: Possesses significant limitations, both in intellectual functioning and adaptive behavior, expressed in conceptual, social, and practical adaptive skills. New Jersey: The United States Department of Education (USDOE) non-regulatory guidance .... The guidance states that: "only students with the most significant cognitive disabilities may be assessed based on alternate achievement standards….the Department intended the term "students with the most significant cognitive disabilities" to include that small number of students who are (1) within one or more of the existing categories of disability under the IDEA (e.g., autism, multiple disabilities, traumatic brain injury, etc.); and (2) whose cognitive impairments may prevent them from attaining grade-level achievement standards, even with the very best instruction." Is the nature of the student's cognitive disability severe? For detailed information about individual criterion specifications refer to Table B8 in Appendix B. As is evident in Appendix Table B8, most states require a combination of criteria for IEP teams to use in making participation decisions for students in assessments. Consortia Participation Criteria AllowedFigure 4 shows the participation criteria mentioned by at least five states and entities in each of the four consortia working on content assessments. It also indicates the percentage of states in each consortia that have each criterion. The number of states in each consortium, as indicated after the consortia name, was the denominator used to calculate the percentages. The top three criteria allowed by states were the same across the four consortia, though in slightly different orders: Has significant cognitive disability, IEP team decision, and Substantial adjustments to curriculum. Figure 4. Number of States and Entities by Consortia that Allow Selected Criteria to be Used for Participation Decisions
Participation Criteria: Not AllowedFigure 5 shows the 14 criteria that states indicated could not be used for AA-AAS participation decisions; this list includes only those criteria mentioned by at least five states. Forty states cited disability label or characteristics as a criterion that could not be used. Language/social/cultural or economic difference (N=31) and extended absences (N=30) were the next most frequently listed. Additional details about the participation criteria that are not allowed are in Table B6 in Appendix B. Criteria that were listed by fewer than five states overall are listed in Appendix Table B7. For example, one state indicated that IQ scores could not be used when making participation decisions. Detailed specification notes for the participation criteria that are not allowed are in Table B8. Figure 5. Number of States and Entities That Do Not Allow Selected Participation Criteria To Be Used in Participation Decisions*
*N=58 Criteria Definitions: Disability label or characteristics: Student's disability category (e.g., Multiple Disabilities, Traumatic Brain Injury, etc.) as well as any descriptive characteristics of challenges a student may have. Language/social/cultural or economic difference: Language, social, cultural or economic status differences. Extended absences: Extended absences from school that may or may not be due to student's disability. Low reading level, achievement level: Student has low reading level or low overall achievement. Expect poor performance on regular assessment: Student is not expected to do well on the regular assessment (or other state alternate). Time in services: Time that student is receiving services, or a particular service. Instruction level of educational placement: The instructional level a student is placed. Anticipate disruptive behavior: Others expect the student will have disruptive behavior. Administration decision: Others in administrative roles make or influence decisions. Impact of scores on school/district: Anticipate that student scores will negatively impact a school or district. Where instruction/services are received: Where a student receives services (e.g., homebound students, etc.) or other program setting. English Language Learner status (ELL) status: Student has English language learner status. Has an IEP: Student has an Individualized Education Program. Expected emotional duress: Student is expected to have emotional difficulties. Consortia Criteria Not AllowedFigure 6 shows the participation criteria that were not allowed, organized by consortia. Again, this figure includes only those criteria mentioned by at least five states. As in other figures, the number of states identified for each consortium (and on which the percentage is based) is the number of states in the consortium. The top three criteria not allowed by states in the consortia were disability label or characteristics, extended absences, and language/social/cultural or economic difference. Figure 6. Number of States and Entities by Consortia that Do Not Allow Selected Criteria to be Used in Participation Decisions
ELL Status and Participation CriteriaSeveral states included information in their AA-AAS participation policies about how the criteria applied to English language learners (ELLs). Often the additional information referred to the federal exemption allowance for the Reading/ELA assessment for new arrivals to the country. In one state, two documents addressing AA-AAS participation criteria were found: one for special education and one for ELLs. Table 1 shows the information provided in these two documents. It is evident in this side-by-side presentation that although some of the information was similar across the two documents, there were some distinct differences. Although the differences may be due to the different years covered by the two documents, they were both the most recent documents available and the ones to which decision makers would probably refer. Table 1. Two Concurrent Eligibility Documents From One State
With the increasing numbers of students with disabilities who are ELLs, and the fact that a small portion of these students are likely to be in the AA-AAS, it is important to attend to the nature of participation policies and criteria for these students. Top of Page | Table of Contents DiscussionThe purpose of this report was to provide information on written participation policies for the Alternate Assessment based on Alternate Achievement Standards (AA-AAS). Our goal was to provide both a national picture of AA-AAS written participation policies, and a snapshot of what these policies look like for each of the two Race-to-the-Top assessment consortia developing a regular assessment, and each of the two General Supervision Enhancement Grant (GSEG) assessment consortia developing an AA-AAS assessment system for students with significant cognitive disabilities. Our review focused primarily on (a) the nature of the participation options (e.g., whether students could participate in the AA-AAS only across content areas, or in both an AA-AAS and another assessment--either the regular assessment or the Alternate Assessment based on Modified Achievement Standards (AA-MAS); and (b) which specific criteria for participation could and could not be used in determining whether a student would participate in the AA-AAS or another assessment. We also noted other characteristics of some of the written participation policies, which we highlight here as well. The information presented in the report was based on 58 states and entities; thus, all but a few participation policies are included in both the national summaries and for each of the consortia. Participation OptionsNationally, written participation policies for the AA-AAS are quite variable in the extent to which they explicitly address the participation options of combined participation (in which a student may participate in the AA-AAS for one content area and in an AA-MAS or regular assessment for another area) and selective participation (in which a student participates in the AA-AAS for one content area and does not participate in any assessment for another content area). Out of the 58 states and entities for which we found written policies that addressed participation in detail, only 20 explicitly addressed either of these options or others (e.g., whether participation can be adjusted for testing window or for disability category). Thus, the information about participation options is limited. In the 20 states that explicitly addressed participation option information in their written policies, over half (n=11) indicated that combined participation was allowed. In other words, a student may participate in an AA-AAS for one content area, and in either a regular assessment or an AA-MAS for another content area. Four states explicitly indicated that combined participation was not allowed. None of the 20 states explicitly indicated that selective participation was allowed, in other words, that a student might participate in the AA-AAS for one content area and then not be assessed at all for another content area, unless there were circumstances such as late arrival in relation to instructional time and the assessment window. Four states explicitly indicated that selective participation was not allowed. The ways in which these participation options were distributed across consortia are difficult to interpret because of the small numbers. Therefore, it is not possible to conclude that one consortia or another has a different perspective from another on the possibility of combined participation, simply because so few states include this information in written policies. The same is true for selective participation. Among the states that addressed whether students' AA-AAS participation might vary as a function of the testing window, seven indicated that participation could be adjusted for the testing window and four indicated it could be adjusted for disability category. For states allowing adjustments for late arriving students, adjustments to the testing window included reducing the number of items completed within a content area, reducing the number of content areas attempted, and allowing a student's file to be submitted without sample work, with or without a petition for exemption based on the time of arrival. Four states indicated that participation could be adjusted for disability category, most often with a focus on students with significant cognitive disabilities who were also blind or deaf/hard of hearing. Because of the small numbers, it is impossible to interpret differences across consortia. It is important for states to include explicit information on participation options in their written participation policies. These are important topics, and should reflect the validity argument that states are making for their AA-AAS. A consistent and well-defined population for which an assessment is designed seems needed if valid inferences about performance are to be made. An information survey of one of the states in one of the consortium focused on the AA-AAS suggested that there are probably unwritten policies generally known by IEP members about participation options, and that these unwritten policies would give a more comprehensive picture of participation options. It seems likely that each assessment consortium will need to have a conversation about their proposed definitions of the population expected to participate in the AA-AAS, the implications of those definitions for participation by content area, and how to handle potential exceptions. Then they will need to develop written participation policies reflecting decisions made during those discussions. Other considerations on participation should also be addressed, including policies on late arrivals (i.e., students who begin attending a school shortly before the testing window) and the transfer of student materials between schools, districts, or states. Only a few states explicitly addressed these kinds of adjustments for participation. It would be useful for states to anticipate in policy what flexibility will be allowed, and what logistical expectations, specific to transfer of student materials, may be set for different types of transfers. Participation Criteria: To Use or Not to UseAll states' written participation policies for the AA-AAS addressed the criteria that could and could not be used for making participation decisions. Criteria mentioned by at least five states were numerous, including 25 criteria that could be used for AA-AAS participation decisions and 14 criteria that could not be used. The top two criteria listed by states were that the student has a significant cognitive disability, and that the decision is made by the IEP team; both of these were included in the written policies of 49 states and entities. The most frequently listed criterion that could not be used was disability label or characteristics (n=40). A small number of states and entities either required IQ scores or mentioned a typical score range as a suggestion for participation decisions. It is important for states to discuss when criteria are requirements and when they are simply suggestions. Not evident from the list of criteria was whether certain criteria had different meanings across the states that included the criteria. Also not evident is whether certain criteria are explicit in some states' criteria while in other states, the criteria are embedded in their IEP documentation forms. Also not evident in the lists of participation criteria in state documents were what seemed to be differences in how the criteria had to be used. Some states required that all criteria be met, while others either listed criteria or described characteristics without explicitly indicating that all must be met. It is not clear how these kinds of differences across states might affect the resulting group of students participating in the AA-AAS. Disability labels and characteristics constituted a criterion that several states indicated could not be used for participation decisions, yet some other states did refer to labels, accompanied by information on specific characteristics typical of students for whom the AA-AAS was designed. It is not clear how differences in participation criteria across states might affect the goal of ensuring that the appropriate population of students participates in the AA-AAS. Several factors will enter into this:
Decisions for ELLsSeveral states did mention English Language Learner (ELL) status of students and how this might affect decisions about participation in the AA-AAS. In some of these states' policies, there was limited mention of ELLs, such as the federal regulation that allows for new arrivals to be exempted from participation in Reading/ELA assessments, or the indication that ELL status could not determine participation in the AA-AAS. Further, it is important for states to be careful in describing participation criteria with regard to language proficiency. For example, referring to ELL status vs. English proficiency could have different nuances for decision-making for students on an AA-AAS. Both terms could refer to ELL status, but clearly, English proficiency also applies more broadly to all students and not only those who qualify for ELL services. Suggestions for the Assessment ConsortiaSeveral of the findings from this study have implications for the assessment consortia developing regular assessments (PARCC and SBAC), alternate assessments (DLM and NCSC), and ELP assessments (ASSETS). For example, all of the consortia will be developing criteria for determining who participates in each of their assessments. States in a consortium will want to discuss exactly where criteria should be placed, and to what extent there should be overlap across different state documents. Further, training and professional development certainly benefit from consistent materials, and this will become of increased importance as the consortia determine their participation policies and the training and professional development to support them. Previous findings have implications for the consortia as well. For example, a comparison of the categories reflected in this analysis of AA-AAS participation policies and the analysis of participation and accommodation policies for all assessments, but primarily the regular assessment (Christensen, Braam, Scullen, & Thurlow, 2009), showed similar participation criteria categories. Still, it is possible that these general criteria may not be specific enough to narrowly define the population of students who should participate in the AA-AAS. This would affect all consortia by possibly introducing gaps in the students who are covered by the participation criteria. LimitationsIssues with terminology were encountered in our examination of states' AA-AAS participation policies. A previous analysis of AA-AAS participation policies noted similar issues with terminology (Musson et al., 2010). To avoid possible misinterpretation, we generally retained the language used by states, with the risk of overlapping meaning in terms or having different results in the number of states with certain criteria due to states using different categorical levels or different levels of detail in describing criteria. We believed that within the context of a single state's policy the actual language used within a written participation policy would be more transparent. Future analyses could examine the relationship between the AA-AAS participation rates and some of the specific criteria that states include in their participation guidelines (see NCEO, 2011). This type of analysis could look at some of the less frequently used criteria, or the definitions of certain criteria (such as "significant cognitive disability") to determine whether certain definitions and criteria result in an increase or decrease in the rate of students participating in the AA-AAS. It would also be important to examine the effects of training on the criteria. Top of Page | Table of Contents ReferencesAchieve. (2010). Closing the expectations gap: Fifth annual 50 state progress report on the alignment of high school policies with the demands of college and careers. Washington, DC: Author. Center for K-12 Assessment & Performance. (2012, March). Coming together to raise achievement: New assessments for the Common Core State Standards. Washington, DC: ETS. Christensen, L. L., Braam, M., Scullin, S., & Thurlow, M. (2011). 2009 State Policies on Assessment Participation and Accommodations for Students with Disabilities (Synthesis Report 83). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Lazarus, S. S., Hodgson, J. R., Price, L. M., & Thurlow, M. L. (2011). States' participation guidelines for alternate assessments based on modified academic achievement standards (AA-MAS) in 2010 (Synthesis Report 82). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Lazarus, S. S., Hodgson, J., & Thurlow, M. L. (2010). States' participation guidelines for alternate assessments based on modified academic achievement standards (AA-MAS) in 2009 (Synthesis Report 75). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Lazarus, S. S., Rogers, C., Cormier, D., & Thurlow, M. L. (2008). States' participation guidelines for alternate assessments based on modified academic achievement standards (AA-MAS) in 2008 (Synthesis Report 71). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Musson, J. E., Thomas, M. K., Towles-Reeves, E., & Kearns, J. F. (2010). An analysis of state alternate assessment participation guidelines. Journal of Special Education, 44(2), 67-78. NCEO. (2011, June). Participation guidelines for new assessments: Thinking through their development (NCEO Brief #3). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Porter, A., McMaken, J., Hwang, J., & Yang, R. (2011). Common core standards: The new U.S. intended curriculum. Educational Researcher, 40(3), 103-116. Shriner, J. G., Ysseldyke, J. E., Thurlow, M. L., & Honetschlager, D. (1994). "All" means "all": Including students with disabilities in performance-based systems. Educational Leadership, 51(6), 38-42. Thurlow, M. L., & Ysseldyke, J. (1997). Large-scale assessment participation and reporting issues: Implications for local decisions. Diagnostique, 22(4), 225-236. Top of Page | Table of Contents Appendix AAA-AAS Documents Used in Review
Top of Page | Table of Contents Appendix B Participation Policies and Criteria by Consortia and States
|
