Table of Contents
- Background
- Day 1
- Welcome from NCEO and the U.S. Department of Education
- Meeting Overview
- Using Interim Assessments to
Appropriately Measure What Students with Disabilities
Know and Can Do: Advisory Panel Takeaways and NCEO Recommendations - State Systemic Improvement Plans (SSIPs) and Assessment-related
State Identified
Measurablel Results (SIMRs) Used for the OSEP Results Driven Accountability (RDA) Initiative - State Discussions
- Alternate Interim Assessments for Students with the Most Significant Cognitive Disabilities
- State Approaches to Interim Assessments
- State Discussions
- Day 1 Closing
- Day 2
- References
- Appendix A: Participants
- Appendix B: Agenda
- Appendix C: Speaker Bios
- Appendix D: Convening PowerPoint Slides
- Appendix E: Discussion Guide for States
Background
More than 147 individuals participated in the Interim Assessment Convening on Measuring What Students with Disabilities Know and Can Do Using Interim Assessments that was hosted by the National Center on Educational Outcomes (NCEO), and held on July 26 and 27, 2021. The purpose of the Convening was to share information about how to enable the participation of all students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities, in interim assessments in ways that support obtaining valid results of what they know and can do. The Convening also provided an opportunity for State education agency (SEA) teams to develop goals and action items, based on the opportunities and challenges identified, that would result in more valid measurement of what students with disabilities know and can do at both the SEA and local education agency (LEA) levels.
Teams from 39 regular states and one unique state participated in the two-day event. The teams included state assessment and special education directors, directors of Title I programs, data managers, LEA representatives, and parents. The state team discussions were supported by 21 facilitators including staff from NCEO and other technical assistance centers. Four participants from the U.S. Department of Education also attended the meeting. Appendix A includes a list of attending state participants, facilitators, and U.S. Department of Education staff.
This Proceedings document was developed to provide a summary of the Convening. It includes appendices and links to resources that were shared by states and others at the Convening.
Interim assessments refer to assessments that are administered several times during a school year to measure student progress. They may be commercially produced or developed by groups of states or other organizations. Other terms that are sometimes used to describe these assessments are local assessments, district assessments, and even formative assessments (although interim assessments rarely provide formative information) (Lazarus, Hayes et al., 2021). Interim assessments are used for a variety of purposes such as measuring growth or progress, making instructional decisions, predicting summative assessment performance, measuring whether a student meets a 3rd grade reading guarantee, or using them as a performance measure for the U.S. Department of Education, Office of Special Education Programs’ (OSEP) Results Driven Accountability (RDA) initiative. There is widespread interest by states and districts in the idea of using interim assessments to measure learning losses that may have occurred because of COVID-19 school closures and inconsistent distance learning. Some states are also interested in potentially using interim assessments as a replacement or supplement to state tests used for accountability.
At the Convening, the takeaways from a February 2021 Interim Assessment Advisory Panel virtual meeting hosted by NCEO were shared, as well as NCEO’s recommendations for enabling all students with disabilities to participate in interim assessments in ways that yield valid inferences about what they know and can do. The Interim Assessment Advisory Panel included psychometricians, researchers knowledgeable about students with disabilities, SEA assessment and special education leaders, LEA leaders, and parents. The purpose of the February Advisory Panel meeting was to develop and disseminate guidance to SEAs on how to more appropriately measure what students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities, know and can do using interim assessments. Advisors at the February meeting noted concerns and gaps about current interim assessments and their uses. They suggested what should be, and proposed practical considerations for a better interim assessment system for each of five areas that comprise a framework: (1) Participation; (2) Accessibility; (3) Role of academic standards; (3) Technical issues; and (5) Data use, interpretation, and reporting ((Lazarus, Hinkle et al., 2021).
The suggestions coming out of the Advisory Panel were summarized in a report published by NCEO: Using Interim Assessments to Appropriately Measure What Students with Disabilities know and Can Do: Advisory Panel Takeaways and NCEO Recommendations (Lazarus, Hinkle et. al. 2021). The Advisory Panel also identified a need for guidance on how to provide alternate interim assessments, so a second publication was produced: Alternate Interim Assessments for Students with the Most Significant Cognitive Disabilities (Browder, et al., 2021). These two publications as well as two related resources provided the foundation for information shared at the Convening:
- The Role of Assessment Data in State Systemic Improvement Plans (SSIPs): An Analysis of FFY 2018 SSIPs (Lazarus, Hayes et al, 2021).
- Interim Assessment Practices for Students with Disabilities (Boyer & Landl, 2021).
The Convening agenda (see Appendix B) was organized around the five-component framework used by the Interim Assessment Advisory Panel. State participants gathered virtually for sharing and presentation sessions, then dispersed to separate breakout rooms to discuss topics and engage in developing action plans for their states. The general organization of the meeting alternated between sharing and action planning.
Two NCEO staff, two state participants, and an external expert provided presentations on topics related to interim assessments and students with disabilities. The presentations of these individuals are highlighted in these Proceedings. Short biographical statements about all speakers are provided in Appendix C. The PowerPoints used for this Convening are in Appendix D.
Participants gathered in the main virtual conference room for sharing and presentation sessions, then moved to individual breakout rooms so that state teams could discuss topics and engage in developing goals and action steps for their states. The general organization of the meeting alternated between sharing and planning. To support the states’ conversations, facilitators used a State Discussion Guide (see Appendix E) that provided space for states to take notes and capture ideas and plans.
This Proceedings document was developed to provide a summary of the Convening. It includes appendices and links to resources that were shared by states and others at the Convening.
Day 1
Welcome from NCEO and the U.S. Department of Education
Sheryl Lazarus, the director of NCEO, welcomed participants, thanking them for their interest and participation in the Convening. She noted that states are grappling with how to include students with disabilities in interim assessments in ways that validly measure what they know and can do. This convening is being held to provide state teams with an opportunity to learn more about interim assessments and students with disabilities, and then to do some planning. She welcomed and thanked the U.S. Department of Education for their support and participation in the Convening.
David Egnor, the NCEO Project Officer at the U.S. Department of Education Office of Special Education Programs (OSEP) then provided his greetings. He thanked Dr. Lazarus and the SEA team members for their interest and participation in the Convening. He acknowledged that the Department knows many states are looking at using interim assessments for a variety of purposes, including to gain a better understanding of the impacts of COVID-19; however, there are challenges related to using interim assessments to validly measure what students with disabilities know and can do. Egnor expressed the Department’s support of NCEO’s efforts to provide states with the technical assistance they need to sort through the issues and develop guidance that will facilitate improved practices for the use of interim assessments with students with disabilities. He also thanked his colleagues from the U.S. Department of Education for their participation and ongoing partnership and support over the years.
Next, Christine Pilgrim, Team Lead for OSEP’s Monitoring and State Improvement Planning (MSIP) Division, expressed appreciation for the invitation to participate, noting that the number of states attending the Convening provided evidence of the interest in interim assessments and students with disabilities. She said that she has also seen much interest in this topic through her work with states on monitoring and state improvement planning. She noted that many states are either currently using, or are considering using, interim assessments as their state identified measurable result (SIMR) in their State Systemic Improvement Plan (SSIP), which is a component of OSEP’s RDA system. Dr. Pilgrim reaffirmed OSEP’s support of states as they worked to include students with disabilities in interim assessments in ways that would more validly measure what they know and can do.
Donald Peasley, Assessment Team Lead, School Support and Accountability (SSA), Office of Elementary and Secondary Education (OESE) at the U.S. Department of Education, thanked NCEO for inviting OESE and thanked states for their participation. He noted that his office and staff had also been learning about states’ interest in interim assessments over the last twelve months.
Meeting Overview
Sheryl Lazarus provided an overview of the Interim Assessment Convening by first providing the purposes of the meeting:
- To share information about how to enable the participation of all students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities, in interim assessments in ways that support obtaining valid results of what they know and can do.
- To provide an opportunity for state teams to develop a set of goals based on the opportunities and challenges identified.
Lazarus next highlighted some of the current and proposed uses of interim assessments (e.g., measuring growth, making decisions about instruction, predicting summative assessment performance, measuring learning loss, measuring whether a student meets a third grade reading guarantee, using as a potential replacement/supplement to state tests used for accountability, and using as a performance measure for the OSEP RDA initiative). She then described the meeting process and provided an overview of the agenda (see agenda in Appendix D).
Using Interim Assessments to Appropriately Measure What Students with Disabilities Know and Can Do: Advisory Panel Takeaways and NCEO Recommendations
In this session Sheryl Lazarus and Andrew Hinkle described the February 2021 NCEO Interim Assessment Advisory Panel takeaways and NCEO recommendations focusing on three groups of students: students with disabilities in general, students with the most significant cognitive disabilities, and English learners with disabilities. The takeaways and recommendations were considered using the previously mentioned five-component framework: (a) participation; (b) accessibility; (c) role of academic standards; (d) Technical issues; and (e) data use, interpretation, and reporting (Lazarus, Hinkle et al., 2021). Lazarus and Hinkle then explained how participants could organize their thinking about interim assessments for students with disabilities by using each of the five components.
Participation
Andrew Hinkle provided information about the first component of the framework: participation. He explained how IDEA requires that all students with disabilities participate in all general State and district-wide assessment programs. IDEA (Section 300.160) states:
(a) A State must ensure that all children with disabilities are included in all general State- and district-wide assessment programs, including assessments described under section 1111 of the ESEA, 20 U.S.C. 6311, with appropriate accommodations and alternate assessments, if necessary, as indicated in their respective IEPs.
Hinkle went on to say that the Every Student Succeeds Act (ESSA) also requires inclusion of all students in assessments used for accountability. For English learners with disabilities, participation requirements are reinforced by several civil rights laws and court cases (e.g., Title IV of the Civil Rights Act of 1964, Lau v. Nichols, 414 U.S. 563 (1974), etc.).
Hinkle noted that some students with disabilities, including some English learners with disabilities, are currently not able to participate in interim assessments because the assessments often lack needed accessibility features and accommodations. He also stated that even though it is required that all students participate in all state- and district-wide administrations of assessments, most interim assessments currently do not have an alternate assessment for students with the most significant cognitive disabilities. Hinkle stated that a recent analysis of 14 commonly used interim assessments did not find any that had an alternate assessment (Browder et al., 2021). He said that SEAs and LEAs should examine their current participation data (in terms of numbers and percentages) to identify the degree to which groups of students with disabilities (e.g., students with sensory disabilities, English learners with disabilities, students with the most significant cognitive disabilities) are excluded from interim assessments.
Hinkle identified key takeaways of the interim assessment advisory panel members regarding participation:
- SEA and LEA interim assessment requests for proposals (RFPs) and requests for information (RFIs) should require interim assessment vendors to provide participation data to the public, including information about participation by student subgroups (e.g., students with disabilities [by category of disability], English learners with disabilities, students who take the assessment with accommodations, etc.). This will provide transparency about who is participating in these assessments.
- SEAs and LEAs writing RFPs and RFIs should ask vendors either to provide an alternate assessment or to develop one. An alternate assessment is required to comply with IDEA and other laws and regulations.
- SEAs and LEAs should work with vendors to merge state or district and vendor files in a way that will allow for ongoing documentation of participation in interim assessments.
- SEAs and LEAs should also communicate with stakeholders and professional organizations (e.g., American Education Research Association Special Interest Group on Inclusion and Accessibility in Educational Assessments) about why it is important for students with disabilities to participate in interim assessments across multiple vendors and platforms. This may include providing evidence of the benefits to students of participating meaningfully in the assessments (e.g., for instructional decision making, measuring progress, access to services for which decisions are based on interim assessments, etc.).
Accessibility
Sheryl Lazarus provided information on the second component of the framework: accessibility. She explained that IDEA requires that SEAs (or in the case of district-wide assessments, LEAs) must develop guidelines for the provision of appropriate accommodations. IDEA (Section 300.160) states:
(b) Accommodation guidelines.
(1) A State (or, in the case of a district-wide assessment, an LEA) must develop guidelines for the provision of appropriate accommodations.
(2) The State’s (or, in the case of a district-wide assessment, the LEA’s) guidelines must—
(i) Identify only those accommodations for each assessment that do not invalidate the score; and
(ii) Instruct IEP Teams to select, for each assessment, only those accommodations that do not invalidate the score.
If an interim assessment is used for federal ESSA accountability, ESSA requirements would also apply. States must make appropriate accommodations available and ensure that their assessments are accessible to students with disabilities (Sec. 1111(2)(B)(vii)(II)). For English learners with disabilities, accessibility requirements are reinforced by several civil rights laws and court cases (e.g., Title IV of the Civil Rights Act of 1964, Lau v. Nichols, 414 U.S. 563 (1974).
Lazarus then described key takeaways of the interim assessment advisory panel related to accessibility:
- RFPs and RFIs should require interim assessment vendors to provide clear evidence that a wide range of accessibility features and accommodations are available. They should also require vendors to provide evidence that available accessibility features and accommodations support valid results and interpretations for the purposes for which the interim assessments will be used. These accessibility features and accommodations should be appropriate and effective for meeting individual student needs.
- RFPs and RFIs should also require application of Universal Design (UD) for assessment principles in all stages of the assessment design and development processes. They should require vendors to provide empirical evidence of how UDA principles have been applied to meet individual student needs and allow for meaningful and valid interpretations and uses of the results and scores.
- It should be emphasized to vendors that they must implement accessibility standards that are compliant with Section 508 of the Federal Rehabilitation Act (e.g., Web Content Accessibility Guidelines1) in their interim assessments.
- SEAs, LEAs, and vendors should provide guidance and training for educators, including those in preservice programs, on selecting, implementing, and evaluating assessment accessibility features.
Role of Academic Standards
Andrew Hinkle provided information on the third component of the framework: the role of standards. He emphasized that federal laws (i.e., IDEA, ESEA) require that all children with disabilities have the opportunity to learn grade-level academic content.2 This includes English learners with disabilities and students with the most significant cognitive disabilities who may need an alternate assessment. ESSA also requires that students with disabilities participate in academic instruction and are tested based on challenging State academic standards for the grade level in which the student is enrolled (Sec. 1111(2)(B)(ii)).
Some interim assessments may not be aligned to a state’s grade-level academic content standards. The purpose for which an assessment is being used affects whether it needs to be aligned to grade-level standards. If the intent is to determine whether students are learning grade-level academic content, the interim assessment must be aligned to the standards.
Hinkle then described key takeaways of the interim assessment advisory panel related to the role of standards:
- Vendors should demonstrate how both general and alternate interim assessments map onto state grade-level content standards if the purpose of the interim assessment is to assess student learning of grade-level academic standards or prediction of student results on state summative assessments.
- SEA and LEA staff who select assessments need to have a good understanding of how standards, instruction, and assessment all work together. They should consider providing resources (e.g., training and support) to help improve the linkages among teaching, learning, and assessment.
Technical Issues
Sheryl Lazarus provided information on the fourth component of the framework: technical issues. She explained a state’s success with its assessment system hinges on the steps it takes to develop and implement a technically sound state assessment system.
The federal peer review process is used for state summative assessments used for accountability. This process results in assessments that are technically stronger. Historically there has sometimes been less emphasis on the technical soundness of interim assessments, but as they increase in profile and are used for new purposes, there is recognition that it is important to consider technical issues for these assessments.
Test scores should have the same meaning for a student with disabilities as they do for other students. In order to draw valid inferences from an assessment, it is important that an assessment produce valid interpretations for the purpose of the assessment. The fairness and equity of an assessment, the comparability of different forms (including alternate formats such as braille), the appropriateness and effect of various accessibility features and accommodations, and other issues related to the validity of score interpretations and uses, all merit evaluation and documentation. The needs of students with disabilities, including students with sensory disabilities, students with the most significant cognitive disabilities, and English learners with disabilities, should be considered throughout the interim assessment development process.
Particular attention needs to be given to validity evidence for interim assessments. Frequently cited evidence of technical adequacy includes item statistics, reliability and measurement error, differential item functioning, factor analysis, linking and equating, and correlation studies.
Lazarus then described key takeaways of the interim assessment advisory panel related to technical issues:
- Interim assessment vendors should use the Standards for Educational and Psychological Testing (AERA/APA/NCME, 2014)3 when developing assessments and providing evidence of technical adequacy and validity.
- SEA and LEA RFPs and RFIs should require vendors to provide detailed information about the technical elements of the interim assessments they are considering purchasing, including evidence (e.g., via the use of think alouds or cognitive laboratory studies) that the assessments are eliciting performance on the intended constructs from students with disabilities. Students with disabilities includes (but is not limited to) students with sensory disabilities and students with the most significant cognitive disabilities.
- Public confidence in interim assessments can be improved with transparency. Knowledge of assessment development processes would allow SEAs and LEAs to communicate with stakeholders about the validity of assessment results, interpretations, uses, the reliability of scores, and the assessment accessibility features and accommodations options available for groups of students with disabilities and English learners with disabilities.
- Detailed information on the technical and validity elements of alternate interim assessments should also be required.
Data Use, Interpretation, and Reporting
Andrew Hinkle provided information on the fifth component of the framework: data use, interpretation, and reporting. IDEA requires that SEAs and LEAs must report assessment data for students with disabilities to the public with the same frequency and in the same detail as other students. This includes reporting on participation and performance in regular and alternate assessments, as well as the number of students participating with accommodations.
A state is eligible for IDEA funding only if it provides assurances to the Department of Education that it takes the following steps:
(D) REPORTS. - The State educational agency (or, in the case of a districtwide assessment, the local educational agency) makes available to the public, and reports to the public with the same frequency and in the same detail as it reports on the assessment of nondisabled children the following:
- The number of children with disabilities participating in regular assessments, and the number of those children who were provided accommodations in order to participate in those assessments.
- The number of children with disabilities participating in alternate assessments.
- The performance of children with disabilities on regular assessments and on alternate assessments (if the number of children with disabilities participating in those assessments is sufficient to yield statistically reliable information and reporting that information will not reveal personally identifiable information about an individual student) compared with the achievement of all children, including children with disabilities, on those assessments (20 U.S.C. § 1412(a)(16)).
SEAs and LEAs should be aware that data from interim assessments are sometimes used for purposes for which they were not intended to be used. Stakeholders (e.g., LEAs, school-based staff, parents, IEP teams) are often unaware of data limitations and contextual factors when using interim assessment data for decision-making purposes. For example, although some interim assessments may be accurate at predicting performance on summative assessments for most students, those with disabilities are often excluded from the assessment development process. These students are often excluded from participating in the assessments due to lack of accessibility. In addition, some students with disabilities are included in interim assessments, but due to lack of appropriate accessibility features and accommodations or alternate assessments, their test results are not a valid measure of what they know and can do. Thus, these scores are not accurate at predicting performance on summative assessments.
Other complexities also occur. For example, sometimes SEA staff are unable to provide accurate guidance to stakeholders about the use of interim assessment data because of the wide variety of interim assessments from which LEAs can choose. It is problematic when interim assessment data are used to draw conclusions across multiple interim assessments without attention to differences in scales, number of achievement levels, and the labels used for those achievement levels. Finally, vendors and LEAs sometimes link their interim data to statewide summative data even though this may not be appropriate.
Hinkle then described key takeaways of the interim assessment advisory panel related to technical issues:
- Test users and vendors should engage in purposeful coordination and collaboration to ensure appropriate data use, interpretation, and reporting.
- SEAs, LEAs, and vendors can build confidence in interim assessment score reports by including information about both the appropriate and inappropriate interpretations and uses of the reported data as well as data limitations, and by showing they used a thoughtful, deliberate process for developing the reports.
- SEAs, LEAs, and vendors should also provide guides to understanding assessment results to educators, families, and other stakeholders. SEAs can post resources on their websites that show how interim assessments fit into the larger assessment system.
SSIPs and SIMRs used for the OSEP RDA Initiative
Sheryl Lazarus next shared information about State Systemic Improvement
Plans (SSIPs)
and Assessment-Related State Identified Measurable Results (SIMRs) used
for OSEP Results Driven Accountability (RDA) Accountability and how
interim assessments play a role in some state RDA initiatives.
In 2014, OSEP introduced a new federal accountability framework, RDA, to monitor and support states’ implementation of the IDEA. As part of RDA, states were required to develop a SSIP, which is a comprehensive, multi-year plan designed to improve outcomes for children with disabilities; and within this plan, to commit to improving a SIMR focused on student outcomes.
An NCEO analysis of states’ FFY 2018 SSIPs found that 43 of 60 regular and unique states have an assessment-related SIMR. A few of these states include interim assessments as the SIMR while other states include interim assessments in the SSIP evaluation plan as a measure of progress (Lazarus et al., 2021).
State Discussions
State teams then went to breakout rooms to discuss their states’ priority areas and action plans related to the critical implementation elements. Facilitators led participants through their discussion guides (see Appendix E) to provide structure for the state teams to think and talk about interim assessments. After introducing themselves to one another and identifying volunteers to be recorders, state teams addressed two questions, as well as other topics that were relevant to their specific state contexts:
- How are interim assessments currently being used in your State (e.g., to measure progress, to measure learning loss, for instructional decision making by LEAs, for a 3rd grade reading guarantee, in the SSIP as the SIMR for OSEP accountability, in the SSIP evaluation plan as a measure of growth, etc.)? Are additional uses under consideration?
- If interim assessments are not used presently in your State, are they under consideration? If interim assessments are presently used in your State, are new or additional interim assessments being considered? If so, for what purposes?
Alternate Interim Assessments for Students with the Most Significant Cognitive Disabilities
Following the state team discussions and after participants were returned to the main room, Diane Browder, PhD, Distinguished Professor Emerita of Special Education at the University of North Carolina at Charlotte (UNC Charlotte) provided a pre-recorded video presentation she had prepared for the Convening (Browder, 2021).
Browder spoke about assessment options for students with the most significant cognitive disabilities, the lack of alternate interim assessments, developing alternate interim assessments, and shared preliminary guidance for development of alternate interim assessments for students with the most significant cognitive disabilities. She stated that IDEA requires that an alternate assessment be developed for each state and districtwide assessment. Browder explained that leaving a group of students out of a key component of an assessment system risks leaving them out of educational decisions and resources as well.
Different interim alternate assessment models will emerge depending on the intended use. Browder used the four levels of interim assessments described by Boyer and Landl (2021) when discussing possible types of alternate interim assessments. Browder described the four levels identified by Boyer and Landl as well as possible adaptations for alternate interim assessments:
- Level 1 (Summative Domain): This level refers to the use of an interim assessment as a mini-summative assessment with content sampled across content areas (e.g., ELA, math, science). An adaptation for an alternate interim assessment uses the context from a real world activity (e.g., going to a museum) to provide items that address comprehension of informational text, math problems related to context, and related science concepts.
- Level 2 (Subdomain): This level refers to interim assessments that offer information about a content area such as ELA. An adaption for an alternate interim assessment might be a literacy assessment in which short passages are read aloud with corresponding questions to sample a range of skills related to ELA content. It might be structured such as a lesson in which the student makes a variety of responses related to passage (e.g., finds a word, fills in a sentence, and completes a graphic organizer). It may include related informational and literary texts or mixed media.
- Level 3 (Reporting Category/Cluster): This level refers to interim assessments that target a set of skills or standards such as informational text or geometry. Given that this focus is on a subset of standards, there would need to be some prioritization of content. An adaptation for an alternate interim assessment might contain passages and series of questions similar to the AA-AAAS, or one category of mathematics, such as geometry and use of one multi-step problem.
- Level 4 (Focal Skills/Standards): The type of interim assessment measures performance on a narrow set of skills or standards. An adaptation for an alternate interim assessment might be a quick assessment of a high priority skill or standard such as comprehension of one read-aloud passage or solving one math problem. It may include supports such as use of graphic organizer (e.g., for story elements) or pictorial task analysis (e.g., for steps to solve math problem) to show progress toward mastery.
Browder identified the following steps that states and districts can take to develop alternate interim assessments:
- Clarify the purpose and use of the interim assessment for which an alternate is needed.
- Convene a stakeholder group to review proposed purpose and use, and to brainstorm what should be included in an alternate interim assessment and a potential assessment format.
- Work with assessment developers to create a blueprint for the assessment.
- Continue working with assessment developers to establish technical quality consistent with the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2013).
State Approaches to Interim Assessments
Representatives of two states (Florida and Ohio) next provided a summary of how their respective states are approaching the inclusion of students with disabilities in interim assessments.
Florida
Vince Verges, Director of Assessment for the Florida Department of Education, provided an overview of the assessment landscape and legislation in Florida and a description of how these impact measuring what all students know and can do. Recent Florida legislation eliminated some of Florida’s assessments, including progress-monitoring assessments. He noted that despite the elimination of the state requirements for local assessments, Florida districts still find value in interim assessments and progress monitoring, primarily as a driver for improving instruction, but also as accurate predictors of performance on statewide end-of-year assessments. In some ways, districts saw the elimination of state requirements for assessments used for progress monitoring as an opportunity for “self-balancing” assessment systems and maintained the use of interim assessment that they felt provided valuable data.
Florida responded to the pandemic by hiring a 3rd-party company to provide a data and analytics platform to aggregate district interim assessment information across some of the major vendors in this space. These analytics allowed for state-level insights into COVID-19 “unfinished learning,” and the degree to which gaps closed throughout the school year as students returned to brick-and-mortar schools. This information allowed the state to monitor progress across different vendors throughout the year in order to identify and amplify best practices, and to offer more timely support where needed.
Verges noted that Florida’s legislature now requires the Florida Department of Education, in collaboration with the state’s Office of Early Learning, to “procure and require the use of a statewide, standardized coordinated screening and progress monitoring system for the Voluntary Prekindergarten (VPK) Education Program and public schools serving kindergarten through grade 8.”
Among other things, the system must (a) identify students who have a substantial deficiency in reading, including identifying students with characteristics of dyslexia; (b) provide data for VPK program accountability; (c) provide VPK providers, school districts, schools, and teachers with data and resources that enhance differentiated instruction and parent communication; and (d) provide information to the Florida Department of Education to aid in the development of educational programs, policies, and supports for providers, districts, and schools.
Verges next spoke about things states may want to consider regarding accommodations for interim assessments. Considerations include:
- Recognize that existing off-the-shelf solutions vary in the degree to which accommodations are provided for students with disabilities.
- Request that vendors provide a wide range of accessibility features and accommodations, including accommodations for students with sensory disabilities (e.g., refreshable braille, graphic organizers, sign language interpretation, etc.) and English learners with disabilities (e.g., glossaries, translations, etc.).
- Recognize that statutory requirements for a “computer-adaptive assessment” still allows for consideration of accommodations that may look different. For example, current Florida law requires statewide assessments in grades 7-high school to be computer-based, but Florida provides paper-based accommodations, as well as large-print and braille for students who require them.
- Request that vendors provide solutions that allow for tracking and reporting the use of accommodations.
- Acknowledge the potential role for a subcontractor to assist with accommodations.
- Consider offering links to research and guidance, such as the recent NCEO work in this area, including a brief summary of what is being offered.
- Consider offering to investigate the feasibility of developing and implementing alternate progress monitoring solutions for students with the most significant cognitive disabilities. An additional scope of work in this area may need to be incorporated as an amendment to a vender contract because it would have cost implications.
Ohio
Jo Hannah Ward, Director of the Office of Exceptional Children, Ohio Department of Education, spoke about interim assessments from a special education perspective. She described how Ohio has systematically approached interim assessments statewide, as described in the State’s SSIP and SIMRs, and discussed what has worked well as well as challenges.
Ohio is targeting early literacy for their SSIP because the data shows that early literacy is predictive of academic success, school completion, and college and career readiness. Many students with disabilities throughout Ohio read significantly below grade level. Disability status, as well as factors such as poverty, greatly affect the likelihood that a student will drop out of school. Focusing on effective literacy instruction at an early age is crucial in improving Ohio’s educational landscape. Ohio’s SSIP includes a focus on three components: (a) systems (Multi-Tiered System of Support and systems coaching), (b) classrooms (professional development and instructional coaching), and (c) students (curriculum-based measures and state assessments).
Ohio has two SIMRs. The first SIMR is the percentage of students with disabilities scoring proficient or higher on Ohio’s third grade English language arts state summative achievement test. It is focused on students with disabilities and Ohio’s Third Grade Reading Guarantee. The second SIMR is the percentage of all kindergarten through third grade students who are on track for reading proficiency, as measured by state-approved diagnostic reading assessments provides the comparison of all students in grades kindergarten through grade 3 and students with disabilities. The Ohio pilot serves 24 schools across 15 urban, suburban, and rural districts in the state, including over 600 educators and over 8,000 students in preschool through grade 3. These districts and schools use the Reading Tiered Fidelity Inventory as an interim assessment.
In Phase I of Ohio’s SSIP (2015), the Ohio Department of Education and its stakeholders reviewed various data sources and found a significant gap between performance targets and performance on state reading and math assessments for all Ohio students, including those with disabilities. This information, and Ohio’s use of state early literacy policies and initiatives such as the Third Grade Reading Guarantee, as well as the knowledge that early literacy predicts future academic success, led Ohio to identify early literacy as the basis for its SIMRs.
In Ohio’s Phase II report (2016), the state focused on building teachers’ capacities to provide high-quality, evidence-based early literacy instruction and intervention, using, and sometimes modifying, state infrastructure; supporting local school districts as they implemented evidence-based practices; and evaluating implementation activities. The Phase II report included five components of the Early Literacy Pilot: leadership; multi-tiered system of supports; teacher capacity; family engagement; and community collaboration. Ohio emphasized the importance of the connections between the five components.
In Phase III, Year 2 (2018), Ohio reported continued professional learning opportunities at the state, regional, district, and school levels, changes to state and regional infrastructures to increase alignment across state initiatives, and data from the first year of the pilot with Cohort 1 buildings. The first year of implementation saw a slight decrease in both of the early literacy SIMRs but an increase in educator knowledge of literacy instructional practices and improvement in measures of a language and literacy Multi-Tiered System of Supports.
Ohio reported continued professional learning opportunities and changes to state and regional infrastructure to ensure continued support of pilot districts and data from the second year of pilot implementation with Cohort 1 and the first year of implementation for Cohort 2. There were increases in both SIMRs (i.e., grade 3 ELA state summative assessment, grade K-3 performance on state-approved diagnostic reading assessments) for Cohort 1 from Baseline to Year 2. There were also increases in teacher knowledge because of the professional learning series for both cohorts. Finally, there were increases in measures of phonemic awareness for students in kindergarten for both Cohorts and grade 1 for Cohort 1. The Phase III, Year 3, SSIP report also addressed barriers to implementation and plans for accessing technical assistance going forward.
Ward stated that pilot components occurred at each level of the educational cascade. Specifically, pilot districts are implementing a Multi-Tiered System of Supports focused on language and literacy and engaging in systems coaching. At the classroom level, educators are engaged in professional development and instructional coaching is ongoing. Additionally, data are collected at the student level through both curriculum-based measures and state assessments.
While Ohio is nearing the end of the fourth year of their five-year pilot, Ward focused on data from the third year because of data limitations that occurred as a result of the pandemic. This was the third year of implementation for Cohort 1 and the second year of implementation for Cohort 2.
The percentage of students with disabilities scoring proficient or higher on Ohio’s third grade English language arts achievement test (SIMR 1) increased from Year 2 (2017-18) to Year 3 (2018-19). Cohort 1 saw a 13 percent increase in the percentage of students with disabilities who were proficient in reading at third grade. Ward stated that although Ohio is pleased about this improvement there are still less than one third of students with disabilities in Cohort 1 who are proficient in reading, so there is still a lot of work to do. She noted that there has been some variation across years and cohorts in grade K-3 student performance on state-approved diagnostic reading assessments (SIMR 2). For example, for Cohort 2, performance was above the target after Year 1and just below the target after Year 2. Ward said that she expects to see the data continue to increase in Year 3 and Year 4 with sustained implementation efforts.
During the summer of 2020, Ohio developed Restart Readiness Benchmark or interim assessments to assist teachers with monitoring student progress towards the achievement of Ohio’s Learning Standards. Restart Readiness includes content-specific toolkits that provide a standards-based set of resources designed to help educators determine where students are when they return to school after the government-ordered school closures as a result of COVID-19. The toolkits include a variety of resources classroom teachers can use to identify instructional gaps that may have occurred during remote learning. The resources may also be used by students to provide support. Some students used accommodations such as extended time, read aloud, etc., when participating in these assessments. For students with the most significant cognitive disabilities, teachers identified resources provided in toolkits that supported instructional planning focused on Ohio’s extended standards and learning progressions. While the interim assessments are voluntary, districts report receiving useful and meaningful data, so Ohio will continue to support these toolkits and interim assessments for the 2021-22 school year. Ward shared links to the Restart Readiness Assessments and Toolkits and the Ohio State Systemic Improvement Plan for those interested in more information.
Ward noted that Ohio provides evidence-based, supplemental instruction based on students’ needs for those not making adequate progress in the core curriculum. In addition, schools have school-wide universal reading assessments and personnel to help with data management. Another practice that Ward was pleased about was that schools are using a variety of data sources to design intensive reading plans.
Ohio has identified some challenges they wish to address. One challenge is that School Leadership Teams are not consistently defining processes to be used by Grade-Level Teams for supporting students with reading skill deficits. Another is that in Tier 3, Grade-Level Teams are not consistently monitoring Tier 3 (intensive intervention) supports so intensive reading intervention plans are not consistently adjusted based on decision rules.
State Discussions
Following the three presentations by Browder, Verges, and Ward, state teams again went to their breakout rooms where facilitators led a discussion of the following questions:
- What did you hear during the Diane Browder presentation on alternate interim assessments that might inform the work of your State?
- What did you hear during the two State presentations that might inform the work of your State? What worked well? What challenges (e.g., data limitations, lack of alternate interim assessments, etc.) did the presenters face?
- What are one or two key items from today’s presentations or from this team breakout group conversation that your state team should not lose sight of as you think about how your state and its LEAs can better use interim assessments to appropriately measure what students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities, know and can do?
Day 1 Closing
Participants then returned to the main room for the Day 1 closing. Sheryl Lazarus encouraged participants to reflect on what they had discussed. She noted the state teams would work the next day on identifying goals and action steps for using interim assessments that appropriately measure what students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities, know and can do.
Day 2
Welcome and Reflections on Day 1
In the opening Day 2 session, Sheryl Lazarus welcomed participants to Day 2 of the Convening, and thanked everyone for their hard work the previous day. She indicated that there were several questions after Day 1 about inclusion of students with disabilities in interim assessments. As a result, she reviewed the definition of interim assessments, as well as relevant federal laws, regulations, and court cases. She also reviewed the purpose of the Convening which was for SEAs to develop goals and action items based on the opportunities and challenges identified that would result in more valid measurement of what students with disabilities know and can do at both the SEA and LEA levels in their State. Lazarus stated the components of the interim assessment framework presented on Day 1 would be reviewed and that state teams would have substantive time during Day 2 to develop goals and plans. First, the participation and accessibility components of the framework would be reviewed. This would be followed by a breakout session where state teams could develop goals and plans regarding participation and accessibility for interim assessments. Next, the remaining three components of the framework would be reviewed: the role of standards, technical issues, and data use, interpretation, and reporting. It would again be followed by a breakout session. The day would close with whole group sharing, and a discussion of next steps.
Review of Participation and Accessibility
Andrew Hinkle next provided a brief review of the participation and accessibility components of the framework. Attendees were reminded that participation requirements are grounded in law. Based on the input provided by the advisors during the Interim Assessment Advisory Panel meeting, NCEO recommends that SEAs and LEAs:
- Obtain data on the participation of students with disabilities in interim assessments and develop a plan for analyzing those data; and
- Provide high-quality alternate interim assessments for students with the most significant cognitive disabilities. (See Lazarus, Hinkle et al., 2021 for more details.).
NCEO’s recommendations for SEAs and LEAs in regards to addressing accessibility include:
- Provide a wide range of accessibility features and accommodations, including accommodations for students with sensory disabilities (e.g., braille, graphic organizers, sign language interpretation, etc.) and English learners with disabilities (e.g., glossaries, translations, etc.).
- Provide high-quality alternate interim assessments for students with the most significant cognitive disabilities. (See Lazarus, Hinkle et al., 2021 for more details.)
State Discussions
State teams next had the opportunity to spend 40 minutes in their breakout rooms for a facilitated discussion, using their Discussion Guides, with leadership from their facilitators, to think, talk about, and develop goals and next steps that addressed:
- Ensuring participation of students with disabilities, students with the most significant cognitive disabilities, and English learners with disabilities, in interim assessments; and
- Ensuring that interim assessments are accessible for students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities.
Facilitators asked three questions to support discussions and goal setting related to participation of students with disabilities:
- Are some students with disabilities excluded from, or not appropriately included in, interim assessments in your state (e.g., students with the most significant cognitive disabilities, English learners with disabilities, students in separate settings, etc.)?
- Are students in some disability categories less likely to be included in interim assessments (e.g., students with visual impairments or blindness, deaf or hard of hearing disabilities, intellectual disabilities, autism, multiple disabilities, emotional disabilities, specific learning disabilities, etc.) in your state?
- Are students with disabilities not included in interim assessments when they are used for some specific purpose (e.g., as a measure of progress, for learning loss, instructional decision making, 3rd grade reading guarantee, in SIMRs or SSIP evaluation plans, etc.)?
Facilitators also offered questions to guide thinking related to ensuring that interim assessments are accessible for students with disabilities.
- Are a wide range of accessibility features and accommodations available for the interim assessments used in your state, including accommodations for students with sensory disabilities (e.g., braille, graphic organizers, sign language interpretation, etc.) and English learners with disabilities (e.g., glossaries, translations, etc.)?
- Does your state and its LEAs provide guidance on the provision of accessibility features and accommodations for interim assessments?
- Are Universal Design of Assessment (UDA) principles used in all stages of the interim assessment design and development process in your state?
Review of Role of Standards, Technical Issues, & Data Use, Interpretation, and Reporting
After breakout discussions, state teams returned to the main room to hear a brief review from Sheryl Lazarus of the third, fourth, and fifth framework components: the role of academic standards; technical issues; and data use, interpretation, and reporting.
In regards to the role of academic standards, based on the input provided by the advisors during the Interim Assessment Advisory Panel meeting, NCEO recommends that SEAs and LEAs:
- Provide transparency on the degree of alignment (or lack of alignment) to grade-level academic content standards of both general and alternate interim assessments (Lazarus, Hinkle et al., 2021).
For technical issues, NCEO recommends:
- Provide documentation and evidence that interim assessments are properly designed to evoke the intended cognitive processes when administered to students with disabilities Lazarus, Hinkle et al., 2021).
For data use, interpretation, and reporting, NCEO recommends:
- Publicly available guidance is provided on the appropriate and inappropriate uses of data in formats that meet the needs of intended audiences (Lazarus, Hinkle et al., 2021).
State Discussions
Following the review of the three framework components, state teams once again convened in their breakout rooms. They had 55 minutes to discuss and identify goals related to these framework components. Facilitators asked three questions to guide states’ thinking and planning about the role of standards:
- How transparent is the degree of alignment (or lack of alignment) to grade-level academic content standards of both general and alternate interim assessments in your state?
- Is documentation of the alignment of the assessment to grade-level content standards publicly provided?
- Do RFPs and RFIs require vendors to show how general and alternate interim assessments map onto state grade-level content standards, as well as correlations between performance on the interim assessment and performance on state standards-based assessments if they are used for predictive purposes?
Facilitators also posed questions to guide discussion about the technical issues of interim assessments:
- Are the interim assessments used in your State properly designed to evoke the intended cognitive processes when administered to students with disabilities?
- Is there documentation and validity evidence (e.g., think alouds, cognitive labs) that students with disabilities, including (but not limited to) students with sensory disabilities, students with the most significant cognitive disabilities, and English learners with disabilities, are demonstrating the assessments’ intended cognitive processes?
- Is information about the interim assessment development process publicly available?
Finally, facilitators asked three questions to guide states’ thinking and planning about data use, interpretation, and reporting:
- Are the limitations of interim assessment data publicly documented, as well as the appropriate and inappropriate purposes and uses of data from each assessment?
- Are there guides and reports that consider the needs of intended audiences (e.g., district and school leaders, teachers, policymakers, parents, and families)?
- Do interim assessment score reports include information about both the appropriate and inappropriate interpretations and uses of the reported data as well as data limitations?
Whole Group State Sharing
The final session of the Convening provided opportunity for state teams to share their takeaways and goals with one another. Some of the points made by state participants included the need: (a) to work with venders to create more inclusive interim assessments; (b) to increase awareness across SEAs and LEAs about the need to include students with disabilities in interim assessments; (c) to help other stakeholders understand why it is important to include students with disabilities (including students with the most significant cognitive disabilities and English learners with disabilities) in interim assessments; (d) to provide more training at both SEA and LEA levels; (e) to include students with disabilities in interim assessments in ways that provides more valid data, and (f) to consider the purpose of administering the assessment when making decisions about which assessment to use.
Closing and Future Technical Assistance from NCEO
Sheryl Lazarus thanked everyone for their participation, engagement, and rich discussion during the Convening. She said that based on evaluation results, state discussions, and input from the Convening facilitators, NCEO will develop a plan and timeline for follow-up technical assistance that will be made available to all states.
Footnotes
1 Web
Content Accessibility Guidelines (2018, June).
https://www.w3.org/WAI/standards-guidelines/wcag/
2 U.S. Department of Education (2015, November). OSERS Policy Guidance on Free Appropriate Public Education. Office of Special Education and Rehabilitative Services. https://www2.ed.gov/policy/speced/guid/idea/memosdcltrs/guidance-on-fape-11-17-2015.pdf
3 The “Standards for Educational and Psychological Testing” were approved as American Psychological Association (APA) policy by the APA Council of Representatives in August 2013 and were published in the 2014 edition of the “Standards for Educational and Psychological Testing.” The Testing Standards are a product of the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education. Published collaboratively by the three organizations since 1966, it represents the gold standard in guidance on testing in the United States and in many other countries.
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2013, August). Standards for Educational and Psychological Testing. Adopted by the American Psychological Association.
Boyer, M., & Landl, E. (2021). Interim assessment practices for students with disabilities (NCEO Brief #22). National Center on Educational Outcomes and National Center for the Improvement of Educational Assessment. https://nceo.umn.edu/docs/OnlinePubs/NCEOBrief22.pdf
Browder, D. M., Lazarus, S. S., & Thurlow, M. L. (2021). Alternate interim assessments for students with the most significant cognitive disabilities (NCEO Brief #23). National Center on Educational Outcomes. https://nceo.umn.edu/docs/OnlinePubs/NCEOBrief23.pdf
Browder, D.M. (2021). Alternate Interim Assessments for Students with the Most Significant Cognitive Disabilities (video). https://www.youtube.com/watch?v=MChBTuptySk
Lazarus, S. S., Hayes, S. A., Nagle, K., Liu, K. K., Thurlow, M. L.,
Dosedel, M., Quanbeck, M., & Olson, R. (2021). The role of
assessment data in state systemic improvement plans (SSIPs): An
analysis of FFY 2018 SSIPs (NCEO Report 425). National Center
on Educational Outcomes.
https://nceo.umn.edu/docs/OnlinePubs/NCEOReport425.pdf
Lazarus, S. S., Hinkle, A. R., Liu, K. K., Thurlow, M. L., & Ressa, V. A. (2021). Using interim assessments to appropriately measure what students with disabilities know and can do: Advisory Panel takeaways and NCEO recommendations (NCEO Report 427). National Center on Educational Outcomes. https://nceo.umn.edu/docs/OnlinePubs/NCEOReport427.pdf
U.S. Department of Education (2018). A state’s guide to the U.S.
Department of Education’s assessment peer review process.
Office of Elementary and Secondary Education.
https://www2.ed.gov/admins/lead/account/saa/assessmentpeerreview.pdf
U.S. Department of Education (2015, November). OSERS Policy Guidance on Free Appropriate Public Education. Office of Special Education and Rehabilitative Services. https://www2.ed.gov/policy/speced/guid/idea/memosdcltrs/guidance-on-fape-11-17-2015.pdf
Web Content Accessibility Initiative (2018, June). Web Content Accessibility Guidelines. https://www.w3.org/WAI/standards-guidelines/wcag/
Appendix A: Participants
Participating States
| American Samoa | Kentucky | Ohio |
| Arizona | Maine | Oregon |
| Arkansas | Maryland | Pennsylvania |
| California | Massachusetts | Rhode Island |
| Colorado | Michigan | South Carolina |
| Connecticut | Minnesota | Tennessee |
| Florida | Missouri | Vermont |
| Georgia | Montana | Virginia |
| Hawaii | Nebraska | Washington |
| Idaho | New Hampshire | West Virginia |
| Illinois | New Jersey | Wisconsin |
| Indiana | North Carolina | Wyoming |
| Iowa | North Dakota | |
| Kansas | Northern Mariana Islands |
Facilitators
| Trinell Bowman | Andrew Hinkle | Virginia Ressa |
| Sheila Brookes | Carla Howe | Tony Ruggiero |
| Kaylan Connolly | Bill Huennekens | Deb Sauberer |
| Cesar D’Agord | Andy Latham | Carol Seay |
| Melissa Fincher | Kristi Liu | Chris Thacker |
| Kascinda Fleming | Dona Meinders | Sandra Warren |
| Linda Goldstone | Kate Nagle | |
| Susan Hayes | Marcia O’Brien |
U.S. Department of Education Staff
| David Egnor, Office of Special Education (OSEP) |
| Lisa Pagano, Office of Special Education (OSEP) |
| Donald Peasley, Office of Elementary and Secondary Education (OESE) |
| Christine Pilgrim, Office of Special Education (OSEP) |
Appendix B: Agenda
Measuring What Students with Disabilities Know and Can Do Using Interim Assessments
Virtual Convening
July 26 & 27, 2021
1:00 – 4:00 PM EST
Agenda
Day 1
| 5 minutes 1:00-1:05 |
Welcome and Opening Remarks Sheryl Lazarus Director (NCEO) David Egnor Donald Peasley Christine Pilgrim (U.S. Department of Education Representatives) |
| 5 minutes 1:05-1:10 |
Meeting Overview Sheryl Lazarus |
| 40 minutes 1:10-1:50 |
Using Interim Assessments to
Appropriately Measure What Students with Disabilities Know and
Can Do: Advisory Panel Takeaways and NCEO Recommendations Sheryl Lazarus and Andrew Hinkle |
| 35 minutes 1:50-2:25 |
State Reflection on the State of the
State State Team and Facilitator |
| 10 minutes 2:25-2:35 |
BREAK |
| 15 minutes 2:35-2:50 |
Alternate Interim Assessments for
Students with the Most Significant Cognitive Disabilities Diane Browder (Professor Emerita, University of North Carolina – Charlotte) (Video) |
| 15 minutes 2:50-3:05 |
A SEA Approach to Interim Assessments Vince Verges Director of Assessment (Florida Department of Education) |
| 15 minutes 3:05-3:20 |
A SEA Approach to Interim Assessments Jo Hannah Ward Director of Special Education (Ohio Department of Education) |
| 5 minutes 3:20-3:25 |
BREAK |
| 25 minutes 3:25-3:50 |
State Team Discussion State Team and Facilitator |
| 5 minutes 3:50-3:55 |
Overview of Breakout Discussions and
Developing Your Unique Goals Kathy Strunk |
| 5 minutes 3:55-4:00 |
Closing of Day 1 Sheryl Lazarus |
Day 2
| 5 minutes 1:00-1:05 |
Welcome and Reflections on Day 1 Sheryl Lazarus |
| 5 minutes 1:05-1:10 |
Overview of Breakout Discussions and
Developing Your Unique Goals Kathy Strunk |
| 10 minutes 1:10-1:20 |
Review Components 1 and 2: Participation
and Accessibility Andrew Hinkle |
| 40 minutes 1:20-2:00 |
Breakout Discussion: Participation and
Accessibility State Team and Facilitator |
| 10 minutes 2:00-2:10 |
BREAK |
| 15 minutes 2:10-2:25 |
Review Component 3, 4, and 5: Role of
Academic Standards, Technical Issues, and Data Use,
Interpretation, and Reporting Sheryl Lazarus |
| 55 minutes 2:25-3:20 |
Breakout Discussion: Role of Academic
Standards, Technical Issues, and Data Use, Interpretation, and
Reporting State Team and Facilitator |
| 10 minutes 3:20-3:30 |
BREAK |
| 15 minutes 3:30-3:45 |
Large Group Discussion Andrew Hinkle |
| 5 minutes 3:50-3:55 |
Summary of Meeting and Next Steps Sheryl Lazarus |
| 5 minutes 3:55-4:00 |
Evaluation Kathy Strunk |
Appendix C: Speaker Bios
 |
Diane M. Browder, Ph.D., is Distinguished Professor Emerita of Special Education at the University of North Carolina at Charlotte (UNC Charlotte). Dr. Browder has researched and written about assessment and instruction for students with severe developmental disabilities since the 1980s. |
 |
Andrew Hinkle is an Education Program Manager at the National Center on Educational Outcomes (NCEO) where he co-lead’s NCEO’s technical assistance activities with the Center’s director. His focus is on topics related to the inclusion of students with disabilities in assessments including issues related to accessibility on state and district assessments. In this capacity, he oversees the development, planning, operation, and internal evaluation of multiple technical assistance activities that are provided to state and federal agencies, and other stakeholders. Prior to joining NCEO, Andrew worked for 14 years at the Ohio Department of Education managing the inclusion of English learners, students with disabilities, and English learners with disabilities in large-scale assessments, including accessibility, alternate assessments, and graduation requirements. |
 |
Sheryl Lazarus, Ph.D., is the Director of the National Center on Educational Outcomes (NCEO). In this position she addresses special education policy issues related to the inclusion of students with disabilities, English learners, and English learners with disabilities in assessments. Her research and technical assistance priorities include: student participation, accessibility and accommodations, alternate assessments, technology enhanced assessments, and formative assessment. She has expertise in the assessments that comprise the comprehensive assessment system, school reform, school accountability, research design (including cost analyses), data-driven decision-making, rural education, and the economics of education. She has published extensively, including papers in peer-reviewed journals in both the fields of education and economics. |
 |
Kathy Strunk, Ph.D., is an Education Program Specialist at the National Center on Educational Outcomes (NCEO) where she conducts research and provides technical assistance related to special education assessment policies and practices. Kathy’s current emphasis is providing technical assistance to states for implementation of requirements related to the ESSA 1.0% state-level threshold on participation of students in Alternate Assessments based on Alternate Academic Achievement Standards (AA-AAAS). She facilitates NCEO’s bi-weekly, 48-state membership 1% Community of Practice and facilitates NCEO’s intensive technical assistance for states working on the 1.0% requirements. Kathy supports technical assistance for State Identified Measurable Results (SIMRs) and also provides leadership of several NCEO national events. |
 |
Vince Verges has overall responsibility for Florida’s K-12 statewide summative assessment system, as well as the Florida Teacher Certification Examination and the Florida Educational Leadership Exam. He has been with the Department for sixteen years and has been involved in developing Florida’s assessments since 1999. Prior to coming to FLDOE, Vince served for ten years in Florida’s public schools and colleges as an elementary school assistant principal, high school mathematics teacher and adjunct college mathematics instructor. |
 |
Jo Hannah Ward serves as the Director of the Office for Exceptional Children at the Ohio Department of Education. The Office for Exceptional Children provides leadership, assistance and oversight to school districts and other entities that provide differentiated instruction for students with disabilities and gifted students; administers state and federal funds; coordinates and administers programs to improve outcomes for students with disabilities and gifted students; and provides technical assistance to districts and educational agencies around issues of compliance with IDEA. |
Appendix D: Convening PowerPoint Slides
Appendix E: Discussion Guide for States
Measuring What Students with Disabilities Know and Can Do Using Interim Assessments
July 26 & 27, 2021
1:00 – 4:00 PM EDT
Discussion Guide
Introduction
The National Center on Educational Outcomes (NCEO) is pleased to host this Interim Assessment Convening. The meeting objectives are to:
- Assist State Education Agencies (SEAs) in planning how to use interim assessments that appropriately measure what students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities, know and can do.
- Support States in their work with Local Educational Agencies (LEAs) to help them use interim assessments in ways that appropriately measure what students with disabilities, including students with the most significant cognitive disabilities, and English learners with disabilities, know and can do.
During this Convening, State teams will have the opportunity to learn about a framework that can be used to organize thinking about interim assessments. The framework is provided in recognition of federal requirements to include all students, including students with disabilities, in statewide and districtwide assessments. It has five components:
- Participation
- Accessibility and Accommodations
- Role of Academic Standards
- Technical Issues
- Data Use, Interpretation, and Reporting
Participating State teams will be provided with opportunities to discuss how their State is using (or considering using) interim assessments, as well as how to enable all students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities, to participate in interim assessments in ways that yield valid inferences of what they know and can do. By including participants from multiple offices and departments, participating SEAs will be able to tap into the knowledge and perspectives of the various team members. Participants will leave the meeting with goals or action steps that will support the use of interim assessments in ways that appropriately measure what students with disabilities know and can do.
This discussion guide is provided to State teams to reflect on and discuss the presentations they hear throughout the Convening. The guide will help to capture teams’ thoughts and to record initial goals and action steps for using interim assessments that appropriately measure what students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities know and can do.
Breakout Session #1. Introduction to the Discussion Guide, Breakout Group Introductions, and Initial Team Discussions
Preliminary Activities
- Brief introductions of breakout group members with a focus on how the work of each group member will contribute to the group. Have breakout group members indicate whether they have been part of the implementation of interim assessments in their State.
- Identify a volunteer to be the note taker for each State represented in the group.
- Identify a volunteer to be the recorder for each State represented in the group. Suggest that this individual may want to set up a collaboration document (e.g., google docs, etc.) for their State team.
- Identify a volunteer to be prepared to contribute to the debrief at the end of Day 2.
Introductory questions that participants can respond to:
- From my perspective (based on my role), important considerations for my participation in this meeting are…
- From my perspective (based on my role), I can contribute to the team and process by…
1. How are interim assessments currently being used in your State – e.g., to measure progress, to measure learning loss, for instructional decision making by LEAs, for a 3rd grade reading guarantee, in the State Systemic Improvement Plan (SSIP) as the State Identified Measurable Result (SIMR) for OSEP accountability, in the SSIP evaluation plan as a measure of growth, etc.? Are additional uses under consideration?
| |
2. If interim assessments are not used presently in your State, are they under consideration? If interim assessments are presently used in your State, are new or additional interim assessments being considered? If so, for what purposes?
| |
3. What did you hear during the first presentation on the advisory panel takeaways and NCEO recommendations that might help inform work in your State? You may wish to consider the five-component framework to guide your thinking.
| Participation: Accessibility and Accommodations: Role of Academic Standards: Technical Issues: Data Use, Interpretation, and Reporting: |
Breakout Session #2. Alternate Interim Assessments and State Approaches to Interim Assessment
Have a discussion about each of the three previous presentations.
1. What did you hear during the Diane Browder presentation on alternate interim assessments that might inform the work of your State?
| |
2. What did you hear during the two State presentations that might inform the work of your State? What worked well? What challenges (e.g., data limitations, lack of alternate interim assessments, etc.) did the presenters face?
| |
3. What are one or two key items from today’s presentations or from this team breakout group conversation should your State team not lose sight of as you think about how your State and its LEAs can better use interim assessments to appropriately measure what students with disabilities, including students with the most significant cognitive disabilities, and English learners with disabilities know and can do?
| |
Breakout Session #3: Participation and Accessibility
Use these forms to frame your State’s discussion and development of goals or action steps.
Tasks: Develop goals or action steps related to:
- ensuring participation of students with disabilities, including students with the most significant cognitive disabilities, and English learners with disabilities, in interim assessments.
- ensuring that interim assessments are accessible for students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities.
Remember to consider both statewide and districtwide administrations of interim assessments, including interim assessments used for your State’s SSIP.
1. Participation
Questions to Consider
- Does your State have data about the participation of students with disabilities in interim assessments in your State?
- Are some students with disabilities excluded from, or not appropriately included in, interim assessments in your State (e.g., students with the most significant cognitive disabilities, English learners with disabilities, students in separate settings, etc.)?
- Are students in some disability categories less likely to be included in interim assessments (e.g., students with visual impairments or blindness, deaf or hard of hearing disabilities, intellectual disabilities, autism, multiple disabilities, emotional disabilities, specific learning disabilities, etc.) in your State?
- Are students with disabilities not included in interim assessments when they are used for some specific purpose (e.g., as a measure of progress, for learning loss, instructional decision making, 3rd grade reading guarantee, in State Identified Measurable Results (SIMRs) or State Systemic Improvement Plan (SSIP) evaluation plans, etc.)?
- Are interim alternate assessments available for the interim assessments used in your State?
Participation
| No. | Goals/Action Steps | Who Needs to be Involved? |
What would be Initial Steps? |
Importance/Urgency (high, medium, low) |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 |
Other Notes
2. Accessibility and Accommodations
Questions to Consider
- Are a wide range of accessibility features and accommodations available for the interim assessments used in your State, including accommodations for students with sensory disabilities (e.g., braille, graphic organizers, sign language interpretation, etc.) and English learners with disabilities (e.g., glossaries, translations, etc.)?
- Does your State and its LEAs provide guidance on the provision of accessibility features and accommodations for interim assessments?
- Does your State provide guidance and training for educators, including those on Individualized Education Program (IEP) teams and in preservice programs, on selecting, implementing, and evaluating assessment accessibility features on interim assessments?
- Are Universal Design of Assessment (UDA) principles used in all stages of the interim assessment design and development process in your State?
Accessibility and Accommodations
| No. | Goals/Action Steps | Who Needs to be Involved? |
What would be Initial Steps? |
Importance/Urgency (high, medium, low) |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 |
Other Notes
Breakout Session #4: Role of Academic Standards; Technical Issues; and Data Use, Interpretation, and Reporting
Use these forms to frame your State’s discussion and development of goals or action steps.
Tasks: Develop goals or action steps related to:
- the purpose for which the interim assessment is used and the role of academic standards.
- any technical issues related to the use of interim assessments for students with disabilities.
- interim assessment data use, interpretation, and reporting for students with disabilities, including students with the most significant cognitive disabilities and English learners with disabilities.
Remember to consider both statewide and districtwide administrations of interim assessments, including interim assessments used for your State’s SSIP.
3. Role of Standards
Questions to Consider
- What are the standards-based issues that the State and LEAs need to consider when using interim assessments to measure what students with disabilities, including students with the most significant cognitive disabilities, and English learners with disabilities, know and can do?
- How transparent is the degree of alignment (or lack of alignment) to grade-level academic content standards of both general and alternate interim assessments in your State?
- Is documentation of the alignment of the assessment to grade-level content standards publicly provided?
- Do requests for proposals (RFPs) and requests for information (RFIs) require vendors to show how general and alternate interim assessments map onto State grade-level content standards, as well as correlations between performance on the interim assessment and performance on State standards-based assessments if they are used for predictive purposes?
Role of Academic Standards
| No. | Goals/Action Steps | Who Needs to be Involved? |
What would be Initial Steps? |
Importance/Urgency (high, medium, low) |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 |
Other Notes
4. Technical Issues
Questions to Consider
- What are the technical issues that need to be considered when using interim assessments to measure what students with disabilities, including students with the most significant cognitive disabilities, and English learners with disabilities, know and can do?
- Are the interim assessments used in your State properly designed to evoke the intended cognitive processes when administered to students with disabilities?
- Is there documentation and validity evidence (e.g., think alouds, cognitive labs) that students with disabilities, including (but not limited to) students with sensory disabilities, students with the most significant cognitive disabilities, and English learners with disabilities, are demonstrating the assessments’ intended cognitive processes?
- Is information about the interim assessment development process publicly available?
Technical Issues
| No. | Goals/Action Steps | Who Needs to be Involved? |
What would be Initial Steps? |
Importance/Urgency (high, medium, low) |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 |
Other Notes
5. Data Use, Interpretation, and Reporting
Questions to Consider
- What needs to be considered about the interpretation and use of data for students with disabilities from interim assessment in your State?
- What are the limitations of the interim assessment data collected in your State?
- Are the limitations of interim assessment data publicly documented, as well as the appropriate and inappropriate purposes and uses of data from each assessment?
- Are there guides and reports that consider the needs of intended audiences (e.g., district and school leaders, teachers, policymakers, parents, and families)?
- Do interim assessment score reports include information about both the appropriate and inappropriate interpretations and uses of the reported data as well as data limitations?
- What resources may be needed that show how interim assessments fit into the larger assessment system?
Data Use, Interpretation, and Reporting
| No. | Goals/Action Steps | Who Needs to be Involved? |
What would be Initial Steps? |
Importance/Urgency (high, medium, low) |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 | ||||
| 5 |
Other Notes