Alternate Interim Assessments for Students with the Most Significant Cognitive Disabilities

When the U.S. Department of Education issued waivers for administering summative assessments in English language arts (ELA), mathematics, and science for the 2019-2020 school year due to the COVID-19 pandemic, many states and districts used commercially-available interim assessments to understand student performance. Interim assessments fall between formative assessment processes that teachers use to track progress during instruction, and summative assessments that usually are administered as part of state accountability systems (Perie et al., 2007). Interim assessments have the potential to promote instructional planning, evaluate efficacy of various programs or instructional approaches, and predict end-of-year proficiency in order to take action for students at risk of failure (Herman, 2017). In recent years, schools and districts have invested heavily to procure and implement interim assessments (Topol et al., 2013).

The Individuals with Disabilities Education Act (IDEA) and the Every Student Succeeds Act (ESSA) both require states to provide an alternate assessment based on alternate academic achievement standards (AA-AAAS) for assessments used for accountability. IDEA further requires that an alternate assessment be developed for each state and districtwide assessment. Whatever assessments a state or district employs, an alternate assessment is required for students with the most significant cognitive disabilities who pursue alternate academic achievement standards.

This Brief notes the lack of interim assessment options for students with the most significant cognitive disabilities. It offers preliminary guidance for the development of interim assessment options for these students.

Assessment Options for Students with the Most Significant Cognitive Disabilities

Educators have well-established options for students who take alternate assessments at the ends of the assessment spectrum: summative assessments and instructionally-embedded formative assessment processes. All states have developed or procured AA-AAAS that align with their state’s academic content standards for use as summative assessments for students with the most significant cognitive disabilities. Most AA-AAAS are performance-based and administered individually by the teacher (Altman et al., 2010; Thurlow et al., 2017). The ongoing establishment of technical quality for these alternate assessments is being documented (Clark & Karvonen, 2020; Multi-State Alternate Assessment, 2018; National Center and State Collaborative, 2016). The growing technical quality of AA-AAAS supports their inclusion in statewide accountability systems.

At the other end of the assessment spectrum, instructionally-embedded assessment strategies are well-established for students with the most significant cognitive disabilities. For example, task analytic and discrete trial assessments can be used while teaching academic content standards (Browder et al., 2020). Nearly all intervention research on academic content for students with moderate and severe disabilities uses these instructionally embedded assessment approaches. Both the Dynamic Learning Maps consortium (Dynamiclearningmaps.org) and the National Center and State Collaborative (http://www.ncscpartners.org) provide resources for using instructionally-embedded assessments.

In contrast, resources on interim assessment for students who take alternate assessments are not part of these growing assessment resources. In describing assessment options for students with the most significant cognitive disabilities, Wakeman et al. (2014) noted the need for interim assessments for this population. As interest in interim assessments grows, so too must consideration for how to provide this option for students who take AA-AAAS. Leaving a group of students out of a key component of an assessment system risks leaving them out of educational decisions and resources as well. Just as interim assessments help educators promote achievement for students who participate in general assessments, they hold the potential to promote alternate achievement standards aligned to academic content standards for students with the most significant cognitive disabilities.

Lack of Interim AA-AAAS

State and local districts often adopt commercially available interim assessments. Although these resources can support educational decision making, they may negatively affect students with disabilities if not used with consideration to their accessibility and applicability. Boyer and Landl (2021) reviewed publicly available technical documentation for a sample of commercial interim assessments for evidence that: (a) students with disabilities were part of the targeted test population; (b) guidance was provided for students who take alternate assessments; (c) principles of universal design and experts in special education were used in test design, development, and standard setting; and (d) accessibility features were available for students with disabilities.

As a follow-up to that analysis, the National Center on Educational Outcomes (NCEO) conducted an analysis of the availability of alternate assessments for 14 interim assessments. The websites for these assessments were searched for publicly available information during February, 2021. As shown in Table 1, none of the 14 commercially available interim assessment offered an alternate assessment option.

Table 1. Alternate Assessments Available for Interim Assessments

Interim Assessment	Alternate Assessment Available
ACT Aspire	No
Developmental Reading Assessments (DRA)3: Benchmark	No
Dynamic Indicators of Basic Early Literacy Skills (DIBELS)	No
FastBridge Assessments	No
Fountas & Pinnell Benchmark Assessments	No
iReady Diagnostic	No
iReady Growth Monitoring	No
iReady Standards Mastery	No
Istation Assessments	No
NWEA Measures of Academic Progress (MAP)	No
Smarter Balanced Focused Interim Assessment Blocks	No
Smarter Balanced Interim Assessment Blocks	No
Smarter Balanced Interim Comprehensive Assessments	No
Star Reading/Math and Star Growth Monitoring	No

A review of the 2018 State Systemic Improvement Plans (SSIPs) by NCEO (Lazarus et al., 2021) also revealed that students who participate in states’ AA-AAAS are missing out on interim assessment options. The review of the Office of Special Education Programs (OSEP) Results Drive Accountability (RDA) requirements found that only seven regular and unique states (Connecticut, Delaware, Texas, Utah, Wisconsin, Guam, and the Northern Mariana Islands) included alternate assessments in their SSIPs' State-Identified Measurable Results (SIMRs) and, without exception, these SIMRs used the states’ summative alternate assessments.

Further, 35 states with assessment-related SIMRs included an interim assessment in their SSIP evaluation plan (i.e., as a measure of progress toward the SIMR). These states employed commercially-available interim assessments. Because none of these interim assessments have an alternate assessment, students with the most significant cognitive disabilities could not be included in the state’s SIMR.

The findings of the two reports (Boyer & Landl, 2021; Lazarus et al., 2021) suggest that an interim assessment option for students with the most significant cognitive disabilities does not yet exist in states’ overall assessment systems. A scan of the research literature also revealed no reference to interim assessments for students who participate in states’ AA-AAAS or guidelines for their development. Given the importance interim assessments have now when there is an increased need for timely data on student learning and progress, the creation of high-quality alternate interim assessment options for students with the most significant cognitive disabilities requires urgent attention.

Developing Interim AA-AAAS

Purpose and Use

As with all assessments, it is important to articulate why you are testing (i.e., purpose) and how the results will be used. Different interim assessment models will emerge depending on the intended use. Potential uses of interim assessments include:

• To measure progress/growth
• To predict summative assessment performance
• To measure learning loss
• To use as a performance measure for OSEP’s RDA
• To use as a potential replacement or supplement to the state tests used for accountability

Most test developers propose that interim assessments can be used for instructional decision making. Teachers of students with the most significant cognitive disabilities already have a wide range of options to make instructional decisions using informal, teacher-made assessments such as those described by Browder et al. (2020). In the early 2000s, some states used these instructionally-embedded data systems as part of a portfolio model of state alternate assessments (Quenomoen, 2009). Teachers were able to promote alternate assessment outcomes by using their ongoing data to improve instruction (Browder et al., 2005). If the only purpose for the interim assessment is instructional decision making, these informal teacher assessments may be sufficient.

Commercial curricula developed to access grade-level academic content for students with moderate and severe disabilities may also be a resource for instructionally-embedded assessments. These fine-grained measures may focus on responses students make during instruction or test retention at the end of lessons or units and can be helpful in promoting progress for students with the most significant cognitive disabilities. However, these instructionally-embedded assessments may not serve the other purposes of interim assessments. If the interim assessment results are intended to inform local, state, and district decision making, curriculum-embedded assessments designed to support classroom instruction will not be sufficient.

A well-developed interim assessment can also help educators determine whether students are making adequate progress toward the overall academic standard. When doing the fine-grained analysis of instructionally-embedded assessment it is possible to lose sight of the end-goal of the standard. Although the student has done well in instruction, performance on the summative AA-AAAS may be lower than expected if learning does not generalize to the targeted standards. For example, a teacher may be using an adapted novel to teach comprehension and is pleased with how well the student is recalling key characters and the sequence of events. But when the teacher administers the alternate summative assessment with its standardized format for an untaught passage of text and new types of questions, the student produces few correct responses. An interim assessment could help bridge this gap by providing multiple opportunities for students to demonstrate progress toward the expectations defined by one or more academic standards. For example, the interim might offer an untaught passage of text and sample a range of comprehension questions. The result could then reveal whether the student can generalize from the literacy focus of daily instruction and inform subsequent remediation activities. If not, the instructional team could utilize more variety in literature used and teach a broader range of comprehension response to promote generalization. In this scenario, the interim assessment supports the implementation of standards-aligned instructional changes. This type of interim assessment could also help the team assess the impact of disruptions in instruction, not only due to system-wide events such as weather and pandemics, but also due to the individual student’s disruptions such as hospitalizations or family relocation.

Stakeholder Involvement

State alternate summative assessments were developed through productive collaboration between measurement experts, special educators with knowledge of how students with the most significant cognitive disabilities access general curriculum, state administrators who coordinated alternate assessments, teachers who served the target population, parent advocates, and commercial vendors who produced the products. Such a stakeholder group should be used to offer revisions and clarifications to the purpose and intended uses of interim assessment results and offer practical and technical input that supports assessment development efforts. As Quenemoen (2009) noted, there were several iterations of summative alternate assessments to enhance technical quality and alignment with states’ academic content standards. Similarly, key stakeholders brainstormed options that would best serve this population with follow-up development by measurement experts. Interim assessments may be designed to support a variety of intended uses and as a result can differ on several dimensions (Boyer & Landl, 2021). Consequently, it is important that the implications associated with the desired uses of alternate interim assessment (e.g., design features, frequency of administration) be considered in advance and evaluated in light of the purpose for assessment and broader learning goals for this population.

Possible Formats of Interim AA-AAAS

Boyer and Landl (2021) identified four levels of interim assessments that reflect differences in the focal content domain, or the granularity of the target of measurement. Depending on the intended use of results, assessments at any of these levels may be designed to support students in AA-AAAS as shown in Table 2.

Table 2. AA-AAAS Adaptations for Levels of Interim Assessments

Level	Brief Description	Possible Adaptation for Interim AA-AAAS
Level 1. Summative Domain	A mini-summative assessment with content sampled from grade (e.g., ELA, math, science). Approximates the summative alternate assessment with sample items for each content area (e.g., ELA, math, science).	Use context of a real-life activity such as going to the museum. Provide items to comprehend informational text, do math problem related to one exhibit, and science concept embedded in another.
Level 2. Sub-Domain	Offers information about a large subdomain of content area such as reading or writing.	A literacy assessment in which short passages are read aloud with corresponding questions to sample a range of skills related to ELA content. May be structured such as a lesson in which student makes a variety of responses related to passage (e.g., finds a word, fills in sentence, completes graphic organizer). May include related informational and literary texts or mixed media.
Level 3. Reporting Category/Cluster	May focus on important learning goal, big idea of content, or set of related skills/standards.	Focuses on one category of standards such as informational text. Uses passages and series of questions similar to AA-AAAS, or one category of mathematics such as geometry and use of one multi-step problem. Category selected based on some prioritization such as the most critical standards or those most at risk for poor achievement.
Level 4. Focal Skills/Standards	Measures performance on a narrow set of skills/standards.	Quick assessment of a high priority skill/standard such as comprehension of one read-aloud passage or solving one math problem. May include supports such as use of graphic organizer (e.g., for story elements) or pictorial task analysis (e.g., for steps to solve math problem) to show progress toward mastery.

Given the resources and expertise invested in current alternate assessment systems, it might be expedient to sample from these systems to create the interim assessment. That is, depending on the desired use of the interim assessment results, it might be a Level 1 mini-summative assessment for students with the most significant cognitive disabilities. An alternative to simply sampling from the AA-AAAS would be to develop items from all content areas using a real-life activity. After setting up the theme, such as a trip to a museum, students would encounter text, math problems, and science concepts that might be encountered in this context. Or, it may be useful to develop Level 2 interim assessments that offer information about a content area such as ELA. Commercial interim assessments include options such as DIBELS and the DRA3 Benchmark and Progress Monitoring Assessments. For students with the most significant cognitive disabilities, an interim assessment might include an assessment that focuses on multiple aspects of comprehension of text that is read aloud. For example, the content may use the theme of a sports tournament and include a story of an athlete followed by informational text on competition rules. Questions might focus on the purpose of the text and sequence of events in the story. The student may then use a table to identify a player’s best scores, or do other items to sample a range of content standards in ELA.

Level 3 interim assessments target a set of skills or standards such as informational text or geometry. Given that this focus is on a subset of standards, there would need to be some prioritization of content. For example, the priority might be the standards of greatest importance for that grade or ones that students are at most risk of not mastering (e.g., as shown by past alternate assessments). The format might include a theme such as the plot of a garden with multiple questions related to surface area of plot and volume of water bucket.

Given how time consuming alternate assessments can be because they must be individually administered, it might also be better to sample a small set of skills like the Level 4 option. For example, the interim assessment might use one read-aloud passage followed by a set of questions for ELA and one problem in math with a series of questions to perform a multi-step operation. Given that the assessment would need to be individually administered, the assessment could be a brief snapshot of performance requiring about 15 minutes or less per content area. The items might also include some instructional scaffolds such as graphic organizers and pictorial task analyses to indicate student progress toward mastery of the standard.

Technical Quality of Interim Assessments for AA-AAAS

The ideas offered here are a starting point for stakeholder groups to brainstorm the development of interim alternate assessments. Technical quality would need to be established through the application of universal design and sound measurement principles. Similar to summative alternate assessments, it may take several iterations of these interim assessments and ongoing research on their technical quality to develop options that can be used in accountability systems.

Steps States and Districts Can Take to Develop Interim AA-AAAS

1. Clarify the purpose and use of the interim assessment for which an alternate is needed.

a. Potential Purposes. Consider whether the purpose is to measure progress/growth, predict summative assessment performance, measure learning loss, be a performance measure for OSEP RDA, or to be a potential replacement or supplement to the state test used for accountability.

b. Consider Need for Alternate: Consider the purposes for which interim assessments are currently used in the state or school district. What alternate is needed for each of these purposes?

2. Convene a stakeholder group to review proposed purpose and use, and to brainstorm what should be included in an alternate interim assessment and a potential assessment format.

a. Identify Potential Stakeholders. Include teachers of students who take AA-AAAS (including those who address the language development needs of English learners with the most significant cognitive disabilities), parents of students who participate in state AA-AAAS, experts in creating access to general education curriculum for students with the most significant cognitive disabilities, experts in design of alternate assessments, administrators who oversee alternate assessment programs, and others who may have been involved in developing AA-AAAS.

b. Review and Revise Purpose Statement. Review existing summative AA-AAAS for state, instructionally-embedded assessments for teachers use, and purpose or need for a third option interim assessment. Review, validate, refine, or enhance the proposed purpose statement for alternate interim assessment. Include a draft rationale and theory of action for how the assessment will be administered and used to serve the defined purposes, Clarify the assumptions that must hold in order for the assessment to have the intended impact.

c. Make Format and Administration Recommendations. Review Table 2 for potential levels of interim assessment as well as brainstorm an overall format. Be sure to consider the needs of students with the most significant cognitive disabilities who are English learners and students who have sensory disabilities who take AA-AAAS and will take the interim alternate assessment. Suggest desired frequency of administration, standardization requirements, and other test specifications that meet the needs of students with the most significant cognitive disabilities.

3. Work with assessment developers to create a blueprint for the assessment.

a. Confirm Purpose. Review purpose statement developed with stakeholders.

b. Identify Desired Claims. Summarize the claims that the assessment and assessment scores should support. For example, the student is on-track or not on-track to proficiency.

c. Define Assessment Format. Summarize format considerations from stakeholder group and how format will fit purpose. Review how alternate assessment format will differ from existing interims to make it accessible for students with the most significant cognitive disabilities including those who are English learners or have sensory disabilities. Review how the interim assessment will align with standards.

d. Create an Assessment Blueprint. Develop the blueprint to ensure the proposed content and structure fits the intended purpose and is appropriate for students who need an AA-AAAS.

4. Continue working with assessment developers to establish technical quality consistent with the Testing Standards.¹

a. Summarize the Development Process. Create a summary of the development process that can be shared with stakeholders to support the reliability and validity of the interim AA-AAAS.

b. Develop Test Specifications and Requirements Documents. Components should include test specifications that are consistent with the blueprint and format requirements, psychometric requirements for scale development and maintenance, content and bias review processes, item-banking needs, scoring requirements, and reporting requirements.

c. Develop Validity Agenda. Identify any additional sources of evidence that will be required to support the desired use claims (e.g., regular scale maintenance, external alignment studies, standards validation, cognitive laboratories, efficacy research).

d. Develop the Test. Activities include content development and reviews, piloting and field-testing, scale setting, forms selection, standard setting, and report development.

e. Administration Manual. Create a clear implementation manual that can be used by teachers and other educators. Pilot use of the interim alternate assessment.

f. Technical Manual. Specify vendor requirements for providing detailed information on technical quality of the interim alternate assessment that is updated on a regular basis. Make ongoing decisions about use and revisions based on technical quality.

g. Limitations of Use. Clarify the intended uses of the interim alternate assessment and limitations in its use.

References

Altman, J. R., Lazarus, S. S., Quenemoen, R. F., Kearns, J., Quenemoen, M., & Thurlow, M. L. (2010). 2009 survey of states: Accomplishments and new issues at the end of a decade of change. National Center on Education Outcomes. https://nceo.umn.edu/docs/OnlinePubs/2009StateSurvey.pdf.

American Educational Research Association, American Psychological Association, & National Council of Measurement in Education. (2014). Standards for Educational and Psychological Testing.

Boyer, M., & Landl, E. (2021, April). Interim assessment practices for students with disabilities (NCEO Brief #22). National Center on Educational Outcomes and Center for Assessment.

Browder, D. M., Karvonen, M., Davis, S., Fallin, K., & Courtade-Little, C. (2005). The impact of teacher training on state alternate assessment scores. Exceptional Children, 71, 267-282.

Browder, D. M., Spooner, F., & Courtade, G. R., & Pennington, R. (2020). Monitoring and enhancing: Student progress: Getting the most out of data. In D. M. Browder, F. Spooner, & G. R. Courtade, (Eds.), Teaching students with moderate and severe disabilities (pp. 93-113). Guilford Press.

Clark, A. K., & Karvonen, M. (2020). Constructing and evaluating a validity argument for next generation alternate assessments. Educational Assessment, 25, 47-64.

Herman, J. (2017). Interim assessments in brief. Center on Standards & Assessment Implementation. https://compcenternetwork.org/sites/default/files/archive/InterimAssessmentsBrief.pdf.

Lazarus, S. S., Hayes, S. A., Nagle, K., Liu, K. K., Thurlow, M .L, Dosedel, M., Quanbeck, M., & Olson, R. (2021). The role of assessment data in state systemic improvement plans (SSIPs): An analysis of FFY 2018 SSIPs (NCEO Report 425). National Center on Educational Outcomes. https://nceo.umn.edu/docs/OnlinePubs/NCEOReport425.pdf

Multi-State Alternate Assessment. (2018). Multi-State Alternate Assessment 2016-17 technical report. https://www.azed.gov/sites/default/files/2018/08/2016-17%20MSAA%20Technical%20Report%20_ADA_Final.pdf?id=5b748ce51dcb250c70810c01

National Center and State Collaborative. (2016). National Center and State Collaborative 2015 operational assessment technical manual. http://www.ncscpartners.org/Media/Default/PDFs/Resources/NCSC15_NCSC_TechnicalManualNarrative.pdf

Perie, M., Marion, S., Gong, B., & Wurtzel, J. (2007). The role of interim assessments in a comprehensive assessment system: A policy brief. Aspen Institute. http://www.achieve.org/files/TheRoleofInterimAssessments.pdf

Quenemoen, R. F. (2009). The long and winding road of alternate assessments: Where we started, where we are now, and the road ahead. In W. D. Schafer & R. W. Lissitz (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 127-153). Paul H. Brookes.

Thurlow, M. L., Rogers, C., & Lazarus, S. S. (2017). 2016 survey of states: State activities amid evolving educational policies. National Center on educational Outcomes. https://nceo.umn.edu/docs/OnlinePubs/2016StateSurveyReport.pdf

Topol, B., Olson, J., Roeber, E., & Hennon, P. (2013). Getting to higher-quality assessment: Evaluating costs, benefits, and investment strategies. Stanford Center for Opportunity Policy in Education (SCOPE). https://edpolicy.stanford.edu/sites/default/files/publications/getting-higher-quality-assessments-evaluating-costs-benefits-and-investment-strategies.pdf

Wakeman, S. Y., Flowers, C., & Browder, D. (2014). Delivering alternate assessment and student progress monitoring in inclusive schools. In J. McLeskey, N. L. Waldron, F. Spooner, & B. Algozzine (Eds.), Handbook of effective inclusive schools: Research and practice (pp. 363-376). Routledge.