Andrew R. Hinkle and Sheryl S. Lazarus
National Center on Educational Outcomes
William F. Lazarus
Department of Applied Economics, University of Minnesota
The Center is supported through a Cooperative Agreement (#H326G210002) with the Research to Practice Division, Office of Special Education Programs, U.S. Department of Education. The Center is affiliated with the Institute on Community Integration at the College of Education and Human Development, University of Minnesota. Consistent with EDGAR §75.62, the contents of this report were developed under the Cooperative Agreement from the U.S. Department of Education, but do not necessarily represent the policy or opinions of the U.S. Department of Education or Offices within it. Readers should not assume endorsement by the federal government.
Project Officer: David Egnor
All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:
Hinkle, A. R., Lazarus, S. S., & Lazarus, W. F. (2023). Disproportionality in the alternate assessment calculator: A tool for state and local education agencies (NCEO Tool #12). National Center on Educational Outcomes.
This resource was refined and piloted through a collaborative process with the 48 states participating in the National Center on Educational Outcomes’ (NCEO’s) 1% Cap Community of Practice (CoP) during its bi-weekly webinar calls in 2021 and 2022. Although the CoP was formed at the request of states to provide them the opportunity for private state conversations, it was with mutual agreement among the CoP members that this tool should be shared publicly on the NCEO website.
The states participating in the 1% Cap CoP are listed here: Alabama, Arizona, Arkansas, California, Colorado, Connecticut, Delaware, Florida, Georgia, Hawaii. Idaho, Illinois, Indiana. Iowa, Kansas, Kentucky, Louisiana, Maine, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, Montana, Nebraska, New Hampshire, New Jersey, New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina. South Dakota, Tennessee, Texas, Utah, Vermont, Virginia, Washington, West Virginia, Wisconsin, Wyoming.
The 2015 reauthorization of the Elementary and Secondary Education Act, known as the Every Student Succeeds Act (ESSA), includes a 1.0% cap on state-level participation rates in the alternate assessment aligned to alternate academic achievement standards (AA-AAAS). This assessment is for students with the most significant cognitive disabilities. States may apply for a waiver prior to the testing window if they think they will go over the 1.0% cap (34 CFR 200.6(c)(2)).
Part of a state’s waiver application requires verifying and addressing disproportionality in the identification of students with the most significant cognitive disabilities. Disproportionality exists in the AA-AAAS when there are atypical differences in the proportions of participants from a student group who took the AA-AAAS compared to those who took the general assessment. Specifically, the state must provide evidence that it has verified that each local education agency (LEA) that the state anticipates will test more than 1.0% of its assessed students in any subject using an AA-AAAS:
(1) followed the state’s guidelines for participation in the AA-AAAS; and
(2) will address any disproportionality in the percentage of students in any subgroup under section 1111(c)(2)(A), (B), or (D) of the Act taking an AA-AAAS (34 CFR 200.6(d)), consistent with section 612(a)(16) (C) of the IDEA.
These student subgroups include seven racial and ethnic groups (White, Black or African American, Hispanic, Native American or Alaska Native, Asian, Pacific Islander, and Multiracial), socio-economic status (as determined by a student’s eligibility for Free and Reduced Price Meals), and English learners. The state must also provide a plan and timeline with clear, actionable steps and milestones for how the state will address any disproportionality in the percentage of students taking an AA-AAAS as identified through the data provided by LEAs (34 CFR 200.6(c)(4)(iv)). The law does not explicitly define the term “disproportionality” or provide any guidance on methodology for analysis of disproportionality for students taking an AA-AAAS.
|
Excel Tool The Excel tool described in this document can be accessed at: Example: https://nceo.umn.edu/docs/OnlinePubs/DisproCalculatorExampleData.xlsx Blank tool: https://nceo.umn.edu/docs/OnlinePubs/DisproCalcBlank.xlsx Cut and paste the link into the browser if the link does not open. |
This tool is a companion to a National Center on Educational Outcomes (NCEO) brief, Guidance for Examining Disproportionality of Student Group Participation in Alternate Assessments (Evans & Domaleski, 2019a). That brief provides a detailed examination and illustration of disproportionality for students with the most significant cognitive disabilities taking an AA-AAAS. It begins by reviewing a framework with respect to its application to address disproportionality. This is followed by an illustration of specific analytic steps and concludes by providing guidance for further inquiry that links back to the guiding principles in the framework. In addition to the guidance in the brief, NCEO and the National Center for the Improvement of Educational Assessment (NCIEA) produced a second companion resource, a video training module: Guidance for Examining Participation Rates and Disproportionality (Evans & Domaleski, 2019b). The video training module discussed the principles and process necessary to engage in a review of participation and disproportionality on the AA-AAAS. It also provided some examples to demonstrate analytic techniques in Excel with sample data.
A state education agency (SEA) recreated the tool presented in the video and used it to examine AA-AAAS disproportionality in the state and in LEAs in that state when providing oversight. After having success with the tool, the state shared it with NCEO’s 1.0% Community of Practice (1% CoP). Other states adopted the tool, and after some use in the field, collectively the states felt the tool could be improved. With input and feedback from the 1.0% CoP, NCEO revised the tool to create the Disproportionality in the Alternate Assessment Calculator.
The purpose of this tool is to support SEAs and LEAs in examining disproportionality with respect to student group participation in their AA-AAAS. Examining disproportionality with regard to alternate assessment participation is essentially an inquiry into whether certain groups are over- or under-identified as having a most significant cognitive disability. Participation data for groups of interest (e.g., race/ethnicity, economically disadvantaged, English learner status, chronic absenteeism) can be entered for a SEA or LEA. The tool then calculates the risk ratio.
The risk ratio is a way to describe the relationship of observed versus expected proportions and is sometimes referred to as the “relative risk.” It is calculated by dividing the proportion of AA-AAAS participants in a particular subgroup (e.g., race/ethnicity, socio-economic) by the proportion of students in the same subgroup who are non-participants. A risk ratio of 1.0 indicates that the proportion of students in a subgroup who are AA-AAAS participants is the same as the proportion of students in that subgroup who are not AA-AAAS participants. In other words, there is no evidence of disproportionality when the risk ratio is at or very near 1. The tool also calculates a confidence interval around the risk ratios that varies depending on how large that subgroup is as a proportion of the overall student population.
The risk ratio can be calculated using the following steps:
Step 1. See the Data Entry sheet for each content area (i.e., reading/ELA, math, science). Enter participation data for subgroups of interest (e.g., race/ethnicity, economically disadvantaged, English learner status, chronic absenteeism) for the AA-AAAS and the general assessment. To address the small numbers in an LEA, use a participation rate based on multiple years of data. Three years is recommended. If the total number of students is large (e.g., a state population or large urban LEA), a year or two years may be used.
Step 2. Scroll down to indicate which years should be included in the risk ratio calculation.
Step 3. Continue scrolling down and review the results to determine if the difference is meaningful. Details of the calculations are shown in the “Summary” sheet.
Whether the relative risk of disproportionality is above expectations, as expected, or below expectations also depends on the confidence level selected. The confidence level is entered in the cell at the top of the Summary sheet. The confidence level is set to 5% by default. In other words, there is a 95% probability that the actual risk ratio falls within the confidence interval around the estimated risk ratio. The confidence interval shown at the bottom of the sheet is used to quantify how close to 1.0 the risk ratio must be to qualify as “very near.” The confidence interval will be wide for subgroups that make up a small proportion of the population and narrow when the proportion is large.
When the lower end of the confidence interval is greater than 1.0, it indicates that the probability or “risk” of a subgroup student being an AA-AAAS participant is greater than expected. For example, when the risk ratio is 2.0 and the lower end of the confidence interval is 1.8, it indicates that a student who is a member of the subgroup is twice as likely to participate in the AA-AAAS. When the upper end is less than 1.0, it indicates that a subgroup student is less likely to be an AA-AAAS participant than expected. For example, a risk ratio of 0.5 with the upper end of the confidence interval at 0.9, indicates that students in the subgroup are half as likely to be AA-AAAS participants.
This tool does not define or identify “significant” disproportionality. This tool only identifies whether the relative risk of disproportionality is above expectations, as expected, or below expectations.
The Summary Sheet provides a summary of the risk ratio and risk of disproportionality for all three content areas (i.e., reading/ELA, math, science).
If it is desired to conduct this analysis for additional content areas (e.g., social studies) or subgroups (e.g., chronic absenteeism, migrant, refugee, etc.) use the spreadsheet under the “Other” tab.
This tool is intended to provide an easy-to-use template for looking at SEA or LEA AA-AAAS participation data to determine if there may be disproportionality of a group. If possible disproportionality is detected, it can be used as a basis for having discussions about the data.
While there is a lack of an explicit definition or guidance on how to calculate disproportionality for students who participate in an AA-AAAS, this tool provides a starting point for SEAs and LEAs to examine their data and ask questions about their findings. If potential disproportionality is detected, some questions that may be discussed include:
Evans, C., & Domaleski, C. (2019a, August). Guidance for examining disproportionality of student group participation in alternate assessments (NCEO Brief #18). National Center on Educational Outcomes.
Evans, C., & Domaleski, C. (2019b, August). Guidance for examining participation rates and disproportionality. https://vimeo.com/325082455
Link to Federal Regulation (2016, December 8):
https://www.govinfo.gov/content/pkg/FR-2016-12-08/pdf/2016-29128.pdf