NCEO Tool 6: Who Should Participate in Your State’s Alternate Assessment?

Overview

Approximately 30 individuals representing 13 states and assessment vendors participated in a virtual forum on July 14, 2021. The forum provided participants with an opportunity to learn about, and then discuss, universal design as it applied to the instruction and assessment of all students. It was jointly held by the Council of Chief State School Officers’ (CCSSO) Assessment, Standards, and Education for Students with Disabilities (ASES) Collaborative and the National Center on Educational Outcomes (NCEO). Similar forums have been held in the past as a pre- or post-session at CCSSO’s National Conference on Student Assessment (NCSA); however, due to the pandemic, this forum was held virtually as a stand-alone event. As in the past, this forum was designed to offer ASES members a chance to dive into a “hot” topic related to students with disabilities.

Specific goals of the forum were to: (1) explore issues related to equity in education, (2) learn how states are addressing the inclusion of all students and incorporating universal design in instruction and assessments, (3) consider how updated principles of universal design can not only meet instruction accessibility standards but also help students become expert learners, and (4) identify strategies for creating universally designed assessments that present an accurate measure of the knowledge and skills of diverse student populations.

Purpose

The purpose of the forum was to bring together states, universal design and equity experts, and other educational stakeholders to explore the role of universal design in creating equitable classrooms and assessments. Universal design was viewed through an equity lens that is inclusive of underserved and marginalized populations, including students with disabilities as well as racially/ethnically and linguistically diverse students, and socio-economically disadvantaged students. It was believed that these discussions could lay the foundation for continued thinking and acting together by states to conscientiously use universal design in instruction and assessment to help ensure equitable education for all students.

Throughout the forum participants were encouraged to consider:

What does equity mean for all students?
What is the role of universal design in creating equitable learning environments during instruction?
What is the role of universal design in creating equitable assessments?
How can states support improved outcomes for all students?

The agenda for the three-hour forum was as follows:

Welcome and Introductions (Kathleen Airhart, CCSSO; Sandra Warren, Independent Researcher; Sheryl Lazarus, NCEO)

Equity for Each Student: Overview and Issues (Kendall Wilson-Flippin, CCSSO)

State Perspectives on Equity in Education (Sandra Warren, Independent Researcher; Toni Wheeler, Washington State Office of Superintendent of Public Instruction; Shawna Benson, OCALI and Ohio Department of Education; Jayne Bischoff, Wisconsin Department of Public Instruction)

Small Group Discussion #1: Critical Issues Related to Equity in Education

Universal Design During Instruction (Jose Blackorby, WestEd)

Universal Design and Assessments (Sheryl Lazarus, Andrew Hinkle, and Kristin Liu; NCEO)

Small Group Discussion #2: Critical Issues Related to Equity Considerations in Universal Design in Instruction and Assessment

Debrief and Next Steps (Sheryl Lazarus, NCEO; Kathleen Airhart, ASES)

Structure of this Report

This report summarizes both the introductory information provided to forum participants and the facilitated discussions that followed. Summaries of the presentations were developed from notes taken during the presentations and from the presenters’ slides.

Participants were encouraged to comment and discuss freely, with assurances that no individual’s name, nor any state or organization, would be attached to comments made. Complete anonymity of statements was ensured. This led to frank and open conversations.

Forum Welcome and Introduction

Kathleen Airhart (Program Director for Special Education Outcomes/ASES Collaborative Advisor, CCSSO) opened the forum, welcoming participants and making introductions. Host and Moderator, Sandra Warren (Independent Researcher), recognized NCEO and ASES members who had worked together to plan the forum. Warren also provided context for the day. Sheryl Lazarus (Director, NCEO) provided an overview of the forum structure and agenda.

Equity for Each Student: Overview and Issues

Kendall Wilson-Flippin (Director of Equity Initiatives, CCSSO) opened her presentation by explaining that the Aspen Education and Society Program and CCSSO define educational equity as: “…every student has access to educational resources and rigor they need at the right moment in their education across race, gender, ethnicity, language, disability, sexual orientation, family background and/or family income” (p. 3).¹

She continued by sharing some specific commitments that states may consider making in order to address issues of equity. These included:

Prioritize equity
Start from within State Education Agency (SEA)
Measure what matters
Create accountability for equity
Go local
Follow the money
Start early
Engage more deeply
Value people
Improve conditions for learning
Empower student options

Wilson-Flippin emphasized that these commitments are starting points for SEAs. Understanding race and racism is a key part of equity. Developing this understanding is multidimensional and multilayered, and can be expressed on multiple levels. SEAs can measure impacts of racism on students. She suggested looking first at day-to-day impacts on students. Leaders must be mindful of their own biases, welcome accountability, and welcome feedback about harm they may have caused even if unintentional. Institutional racism is reflected in disparate funding, predominance of certain cultural values, and a lack of culturally responsive texts and materials. Structural and societal impacts include policies, systems, and ideologies (e.g., consequences of violence, lack of educational opportunities, etc.).

Wilson-Flippin shared examples of culture shift approaches (e.g., educators examining their bias and role, educators recognizing that racism is systemic). Other examples include monitoring disproportionality for over- and under-identification, using culturally responsive curriculum and assessments, fostering healthy conversations about race and racism, engaging diverse expertise, making sure all students have access to a high-quality education, and recognizing the importance of language. There is a need to find ways to measure progress and evaluate educational equity.

Participants were reminded that educators need to examine themselves and their role, as in the Person-Role-System framework by Kathleen P. White.² For example, “Person: How does who you are as a person affect how you exercise leadership? What feels normative to you? What are things you don’t have to consider daily related to identity?” and “Role: How does your title describe what you do and the influence you have? What is your informal authority? Where do you exercise your primary leadership role? How is success defined?” Wilson-Flippin noted that it is important to understand the system, the context, and the intended results for the system.

Wilson-Flippin made the point that schools often use academic and behavior standards aligned more with white middle class privileged children, while minority children are more likely exposed to biological and environmental factors (e.g., fetal alcohol spectrum disorders, poverty, etc.) that can reduce their academic and behavioral functioning. Minority families may have fewer interactions with professionals who diagnose disorders in early childhood. These may also impact how minority children are identified for special education services. A combination of factors (e.g., color, disability, behavior) impact representation in special education. Care should be used to address disproportionality because the problem may be rooted in a system that has already created a distinction for minority students. Assessment can be a part of inequitable education, yet assessments are often used as predictors of future learning. Wilson-Flippin stated that culturally appropriate and culturally responsive curriculum and assessments are important, and funding should be prioritized to address these issues.

Wilson-Flippin challenged participants to work to reduce inequity by considering themselves as allies and accomplishing what they can within their scope of work. This could include fostering healthy conversations about race and racism, engaging diverse expertise, and making connections to intersecting variables that may bar access to a high-quality education. She reiterated the importance of recognizing the use of language and its meaning because it could make some feel “less than.” She urged participants to move toward action in all aspects of educational equity that address the experiences of students and address oppression, discrimination, and exclusion.

State Perspectives on Equity in Education

Following Wilson-Flippin’s presentation, Sandra Warren, Moderator, introduced representatives from Washington, Ohio, and Wisconsin who presented how their states were using universal design to promote equity and access in education.

Washington

Toni Wheeler (Alternate Assessment Coordinator, Washington State Office of Superintendent of Public Instruction) shared information about what Washington is doing, and presented on Washington State - Science for All. She began her presentation by considering her role as alternate assessment coordinator with regard to equity and access. She explained that access is always at the forefront, whether for the alternate assessment or accommodations for general assessment. In her role, she focuses on where the best instructional practice and universal design for learning live, and seeks opportunity to move thinking.

Wheeler shared that there are challenges in working among department divisions where it may seem that teaching students with the most significant cognitive disabilities is not the responsibility of either the assessment division, learning and teaching division, or special education division. SEA work can seem siloed. Washington’s new general science assessment work began in 2014 (after the adoption of the Next Generation Science Standards [NGSS] in 2013), followed by work on the alternate science assessment in 2016. This created an opportunity to thoughtfully align alternate assessment science expectations to the general science expectations, thereby increasing access to science instruction for students with the most significant cognitive disabilities (as well as other students). This was the first time Washington made an intentional effort to create a solid alignment across the two assessments.

The big question for the Washington SEA was “Can Washington align alternate work with established general assessment work to address the lack of science instruction for students with significant cognitive disabilities and strengthen the instructional link between where any learner is in relation to the grade level instructional expectations?”

Washington’s intentional planning for the Washington Access to Instruction & Measurement (WA-AIM), which is Washington’s Alternate Assessment based on Alternate Academic Achievement Standards (AA-AAAS), involved an understanding that:

The Science WA-AIM must hold true to the three-dimensionality of the Performance Expectations as intended in the NGSS.
The Science WA-AIM must be a collaborative effort by special educators AND science content educators.
The Washington State Office of Superintendent of Public Instruction (OSPI) must also model this practice through assessment, learning and teaching, and content assessment.
WA-AIM content should be informed by Washington Comprehensive Assessment of Science (WCAS) content-feedback loops.
WA-AIM development processes should mirror WCAS processes wherever possible.
The assessment must assume and expect that instructional opportunity in three-dimensional science is being provided to students with the most significant cognitive disabilities.

The backbone of the WA-AIM access point frameworks included the regular state learning standard leading to the essential concept. This is followed by a more complex access point, an intermediate complex access point, and then, a less complex access point. In other words, the regular standard is reduced in depth to the SEA’s essential concept for the standard, or in the case of science, the initial reduction in the NGSS Performance Expectation (PE). From there are the three levels of access points in which students can be measured in relationship to the expected achievement level. The intermediate complex access point is intended to be the expected mastery level of the essential concept.

Wheeler shared that the SEA began with the foundational structure from the state’s general science assessment, and then worked to implement that within the alternate assessment, which has a non-traditional design. In her example, the general science assessment is founded in bundling the NGSS PEs, centering those bundles around a shared phenomenon or design problem. This was followed by the development of two-dimensional and three-dimensional items to ensure full measurement of the PEs. In other words, the SEA centered each measured NGSS PE around a phenomenon or design problem, built those out into their access point frameworks, and then used the idea of a set of items working together as a cluster to fully measure the PE.

The SEA built in processes to the alternate assessment development that involved multiple feedback loops for ensuring closer alignment of the two science assessments compared to their other content assessments. These included:

– Same prioritization of standards

– Access Point Performance Tasks redesigned to follow the regular assessment item cluster approach

– Overlap in participants throughout the development process

– Collaboration across OSPI’s assessment, learning and teaching, and special education

– Overlap in vendor

– Same first operational year of new assessment

This initiative resulted in several successful outcomes: (1) it provided a strong instructional link for students with the most significant cognitive disabilities to the grade-level curriculum that can be applied to all students, (2) it strengthened the relationship between internal agency divisions as well as involved general and special educators, (3) it allowed shared participants to bring a deeper understanding to each assessment’s individual activities, and (4) the SEA modeled cross-divisional collaboration. Particularly important was that the process improved equity by making tests accessible to students through design and development processes, provided test development with consideration for diverse student groups, and provided fidelity to knowledge and skills of measured standards.

Wheeler noted that while the SEA was not “there yet,” several critical outcomes demonstrate a move to increased equity:

Science educators found:

Proactive ways they could reach out to special education teachers to support science content instruction
Usefulness of the resources developed and how these can support any learner in their class

Special educators found:

Renewed acceptance of academic instruction
Establishment of collaborative relationships with general education colleagues

Wheeler is now seeing:

District interest in building out their access point frameworks for all standards and content areas
Districts giving wider access to WA-AIM resources for other instructional purposes

Wheeler concluded with the key points that “instructional equity starts with access to instructional opportunity” and “instructional opportunity needs intentional thought and planning.”

Ohio

Shawna Benson (Program Director for the Teaching Diverse Learners Center, Ohio Center for Autism and Low Incidence [OCALI] and the Ohio Department of Education) presented Systems Level Universal Design Leading to Universal ACCESS. She began with the premise that the goal is for Universal Access – not just Universal Design. She pointed out that barriers are within the learning opportunities, environment, and materials, not the students. In this case, Ohio looked back to consider the systems barriers that were getting in the way of student learning.

Ohio’s plan to Raise Literacy Achievement revolves around universal access. The work involves: (1) integrating reading research with diversity-group specific research, (2) providing subgroup specific implementation examples, (3) diversifying screening and diagnostic assessment access, and (4) enhancing requirements for including and assessing learners with the most complex needs.

Benson shared that a partnership approach is being used with state departments and stakeholders where access is considered at the onset. Ohio identified access barriers such as lack of opportunity, learning environment, and participation in all assessments. Ohio also carried out a root cause analysis on how the SEA contributes to barriers. State personnel are working now on targeting and addressing SEA gaps. Ohio was also able to leverage equity and inclusive practices through the State Systemic Improvement Plan (SSIP).

Ohio’s Access to the General Curriculum work included a new alternate assessment decision-making tool that is used to identify students who should appropriately participate in the alternate assessment, state collaboration with 16 regional liaisons for professional development and coaching, and the development of new online support resources which are available at https://literacyaccessforall.org/.

Wisconsin

Jayne Bischoff (Universal Design Education Consultant, Wisconsin Department of Public Instruction) presented Wisconsin Universal Design for Learning. The state is working to improve equitable education for unserved and underserved learners, students with the most significant cognitive disabilities, English language learners, students of color, and often those with or without a particular label. Universal Design for Learning (UDL) is a statewide initiative that is intended to be a unifier of many initiatives across the department, such as career- and college-ready Individualized Education Plans (IEP), equitable multi-tiered systems of support, and local education agency (LEA) determination.

The goals of Bischoff’s presentation were to: (1) explain how the state’s UDL discretionary grant project is a means to support Wisconsin’s statewide implementation of UDL, and (2) identify features of the Wisconsin vision for UDL and equitable educational opportunities for each person.

Bischoff named two systems of support for UDL implementation: (1) the equity mindset that UDL requires (the onramp to applying the UDL guidelines), and (2) a formative assessment resource that helps educators remove or reduce barriers in formative assessments.

Wisconsin’s scaling up project involves supporting Cooperative Educational Service Agencies (CESA) at the elementary, middle, and high school levels in rural, suburban, and urban areas. There are 12 CESAs plus nine UDL Demonstration Sites in the state. The state’s discretionary grant project has aimed to support UDL implementation throughout the state through training and follow-up coaching from education consultants at the state’s CESAs, through site visits and presentations by designated demonstration site leaders who share their own journey in implementing UDL, and systematizing UDL. Bischoff said that the state’s professional educators and their systems are developing expertise in continuous learning, and in the very act of learning itself. A goal of UDL is to develop expert learners, and in Wisconsin that means developing expert learners, expert educators, and expert systems.

Wisconsin designed its UDL grant project using the science of implementation and the five active frameworks from the National Implementation Research Network (NIRN). This includes UDL linked teams, stage-based supports, implementation drivers that manifest in a benefit to students, and cycles of improvement that inform fidelity and impact. Bischoff made the point that, “We can hope implementation happens, we can let it happen, or we can make it happen by design.”

Bischoff said that the formative, classroom application of UDL is studied through instructional rounds, site-visit observations, and UDL implementation trials. An academic focus area and engagement characteristics are part of team work to design learning experiences, apply UDL principles and guidelines, and use formative assessments with generalizable scoring tools to establish a baseline and subsequent data points. Bischoff shared that Wisconsin is using an online unit on eliminating barriers in formative assessments as part of an Assessment Literacy Training Module set.

In Wisconsin, UDL is a principled approach to educational justice. It requires “learning design” as compared to “lesson planning.” This required flexibility, addressing barriers to variability in the design and setting clear goals with options that matter (e.g., options that are accessible, usable, and beneficial to each person). Bischoff emphasized that UDL operationalizes equity. It gives educators strategies for improved equity they can implement immediately. This contributes to a conscious awareness of the environment and the design of learning that limits expectations and outcomes for individuals.

Bischoff closed by providing resources for participants:

Small Group Discussion # 1: Critical Issues Related to Equity in Education

After the presentation by Kendall Wilson-Flippin on equity and the three presentations by state representatives, forum participants were assigned to one of three breakout rooms to engage in a small group discussion of three questions. A summary of these discussions is provided here.

1. What did you hear from Kendall Wilson-Flippin or the three state perspectives that is relevant to your work regarding equity for each student? Given the current climate, how difficult is it for you to do this work in your state?

Confusion of what can be taught (for instance, legislation preventing the teaching of critical race theory).
It is hard to challenge colleagues and friends when we see marginalization happening.
Building relationships is important and equity work is more digestible when working in a safe, non-judgmental environment.
Educators need to be reflective and introspective.
For change to happen it takes the work of individuals, teams, and communities to build trust and share mindsets, values, and beliefs.

2. What are the barriers to inclusion of students with disabilities and other underserved, marginalized populations in equitable classrooms and assessments?

Need to keep focus on students with disabilities, including students with the most significant cognitive disabilities. We need to be intentional that “all means all.”
Debate on what is culturally responsive teaching.
Difficult to get a diverse group for standard setting and content, and bias reviews for test development.
Not everyone is on board with equity, including state boards of education, local and state leaders, parents, educators, administrators, etc.
Assessments are not culturally responsive.
Not everyone is at the table when SEAs are making system-level critical decisions.
In education, we can be reactive to new and current events that we are passionate about, but we do not take the time or have the capacity to design how we want to solve a problem. Using a theory of change or action model may help states formulate where they want to go.
Language unintentionally creates barriers.
Reliance on 1:1 paraprofessionals for instruction rather than facilitating access.
Institutions of higher education fail to address this in their programs of study.

3. What promising practices have we begun to see over the last year to reduce barriers to inclusion?

One state reported it has an inclusion initiative for students with the most significant cognitive disabilities.
Physically moving SEA teams (e.g., special education consultants could be housed within curriculum, English learner specialists could be embedded with SEA teams, etc.).
Assessment system needs are beginning to be addressed.
Resources for LEAs are improving.
Mindfulness about the amount of change people can handle, thinking about capacity to do it well, using a thoughtful process.
Adding an equity lens when developing and editing materials for publishing.

Whole Group Discussion
Participants were moved to the main virtual room following the first breakout session. Sandra Warren, Moderator, invited participants to share key takeaways from their discussions with the whole group and thanked participants for sharing their thoughts and ideas.

Universal Design During Instruction

Jose Blackorby (Research Director, Learner Variability and Impact, WestEd) presented UDL in Instruction. He emphasized the rapidly changing nature of the world, including the skills now required for jobs and democracy. He made the point that COVID-19 has only hastened this change. He noted that since the beginning of the pandemic students may not have been connected to school at all, may not have had opportunity to learn, and may have been affected psychologically and socio-emotionally. Everyone is exhausted. He commented that flaws in systems and practices have been uncovered, everyone wants answers, and everyone wants to make things better going forward. It is timely to discuss UDL in this context.

During a CAST conference held in the summer of 2020, discussions were held about many kinds of barriers that current UDL guidelines may not address well. This led to the development of a draft second version of the guidelines that examines UDL through a lens of equity.

Blackorby went on to describe a graduate degree course he taught at Harvard University, Graduate School of Education, where the goal was to create the best laboratory of what UDL can look like at the secondary level. He taught the course even before the pandemic. Students innovated, tried things, failed occasionally, and adapted. The course had a capstone project that included a combination of readings, audio, and video. When the pandemic began, the course had to pivot to online learning and be rebuilt. The new design principles were simplified and focused on what was most important. This included clarity (what seemed clear to instructors may not have been clear to students), community (the goal was to create a virtual community), equity (the goal was to incorporate equity every week instead of in a few sessions), engagement, and choice.

Blackorby described numerous class activities that included students from 15 countries and many time zones. These included the creation of short videos by the students where they told the others about themselves, the creation of a map with dots so the group could see where people were from, and the use of a tool that highlighted the personal interests of the class. Students were invited to tell instructors how they learned best, and norms were developed that were implicit rules from the perspective of the students. The class used an online whiteboard with “sticky notes” to develop the norms. Small groups of students could group things together that were important to them. The notes were grouped and simplified into six categories that were used every week to evaluate the class.

Every class began with a centering activity (poem, thought, music, etc.) to get everyone in the same frame of mind. Harvard had a lot of available technology for students that Blackorby wanted to use, but the students wanted to keep things simple, so they used a few tools well and consistently. Flexibility was required because some things common in the United States were not available in other countries. The course used reflections which the students suggested making more central to the course than originally planned. This led to making the student reflections key components of the content of the course. A variety of materials (cartoons, mind maps, graphics, etc.) were also used. Authentic reaction to the content also became the content of the course. An escape room was built around learning objectives related to UDL. Students participated in design challenges. For example, they received clues to a riddle that they had to figure out. The course had elements of a game room, popular culture, fun, entertainment, and good instruction.

Blackorby observed that with UDL, we make everything accessible, but at the same time we need to remember that students are the expert learners. We should incorporate student choice, listen to students and respond to their feedback, be innovative, accept that failure happens sometimes, and remember to adapt. In closing, Blackorby emphasized that educators should strive to design learning environments where differences are not impediments to learning.

Universal Design and Assessments

Sheryl Lazarus, Andrew Hinkle, and Kristin Liu representing NCEO, presented Universal Design and Assessments, beginning with Lazarus (Director, NCEO) explaining NCEO’s recent work to update the 2006 State Guide to Universally Designed Assessments. A draft version was shared with forum participants. The update will reflect today’s assessments, addressing the inclusion of all students, including students with disabilities, English learners, and English learners with disabilities. It considers linguistic and cultural factors. The purpose of the guide is to provide states with strategies for designing tests that create an accurate measure of the knowledge and skills of diverse student populations. She welcomed comments on the draft version.

Lazarus reminded participants that the 2015 Every Student Succeeds Act (ESSA) directly applied the concept of universal design to assessments; it specifically included alternate assessments. States must now use the principles of UDL when developing or revising their assessments, aiming to incorporate assessment accessibility to improve overall fairness and opportunity for students to show what they know and can do. Lazarus shared the principles of universally designed assessments:

Universally designed assessments do not change the standards measured by the assessments.
Universally designed assessments should improve the validity, reliability, and fairness of the assessments.
Universally designed assessments may reduce the need for testing accommodations.
Universally designed assessment may improve the validity of assessment results and interpretations for all students, including students with disabilities, English learners, and English learners with disabilities.

Lazarus said that elements present in universally designed assessments: are inclusive of the student population (all students participate); have precisely defined constructs and accessible, non-biased items; are amendable to accommodations; and have simple, clear, and intuitive instructions and procedures.

Andrew Hinkle (Education Program Manager, NCEO) presented steps for applying universal design to assessments. These include:

planning for universal design from the start,
defining the test purpose and approach,
requiring universal design in Requests for Proposals (RFPs),
addressing universal design during item development,
including universal design expertise in review teams,
performing usability and accessible testing,
implementing item and test tryouts,
conducting item and test-level analyses and acting on results, and
monitoring test implementation and revising as needed.

Kristin Liu (Assistant Director, NCEO) shared some cultural and linguistic diversity considerations. These include:

States need to be aware of the characteristics of their student populations because each state has unique demographics.

Populations of students with disabilities differ across states in the percentage identified in certain disability categories, and
The language backgrounds of English learners vary across states.

Planning requires the recognition that the abilities, linguistic profiles, and sociocultural backgrounds of students vary greatly, and assessments need to be accessible to all students.
Needs of diverse learners should be considered when developing both general and alternate assessments.

In the construction of test items, it is important to involve everyone in the process to ensure that content that could advantage, disadvantage, or introduce offensive content for any student is not included. For example, items must be sensitive to test-taker characteristics and experiences, such as gender, age, ethnicity, socioeconomic status, religion, disability, or language, as well as the needs of students who use assistive technology. Items should minimize the effects of extraneous factors, such as avoiding the unnecessary use of graphics that cannot be presented in braille or using font size and white space appropriate for clarity and focus. The readability and language complexity of reading materials should not exceed grade-level expectations.

Liu emphasized that advisors and stakeholders have a key role which includes involving sensitivity teams to examine assessments for items that might introduce bias or offensive content. They should also examine how particular identity groups are portrayed, and whether linguistic or experiential biases may provide an advantage unrelated to test constructs. Educators and representatives from diverse identity groups can help identify potentially biased items. Accessibility and accommodation experts can also be part of the sensitivity review teams to help provide expertise on biases that may be introduced or accessibility barriers causing items to function differently for different populations.

Small Group Discussion #2: Critical Issues Related to Equity Considerations in Universal Design in Instruction and Assessment

Following the presentations on universal design in instruction and assessments, participants were again assigned to their original small group to discuss four questions.

1. What did you hear from the first two presentation sections (Kendall Wilson-Flippin and the Washington, Ohio, and Wisconsin Perspectives) that are relevant to the presentations you just heard on universal design of learning and assessment?

A connector across the sessions was the common thread about the importance of getting multiple perspectives including those of stakeholders and community members.
Equity is connected to test items and curriculum. We need to keep in mind how this affects all populations of students.

2. What did you hear from the presentations on universal design of learning and assessments that resonated with you and why?

During test development we may need to conduct different types of data analyses and use different item development processes than have historically been done.

3. What are the challenges and opportunities related to implementation of equity-based universal design of learning and assessments? How are these related?

The state has reported that many students are multi-ethnic, in that they may identify with one culture more than another.
Students do not fit in the federal disability categories.
Translations for assessments are difficult because there are so many languages.
Due to the pandemic, we are unable to hold cognitive laboratory studies to test new item designs.
There are a low number of students taking the alternate assessment and states share test items with other states; this can create loose alignment with state standards.
There are concerns about accessibility of some items on general math assessments for students who are blind or visually impaired, as well as for blind or visually impaired students with the most significant cognitive disabilities who take the AA-AAAS.
It is important to take the opportunity to intentionally examine what is working and what is not working, and then continually revisit to make iterative improvements.
With all the diverse representations of UDL there may be situations when things that make an assessment accessible for some students can cause barriers for other students. How do we keep our eye on universal access?

4. What resources can CCSSO and NCEO offer to support your work?

Verbiage and specific bullet points for requests for proposals (RFPs) that can be used when working with vendors.
A state guide for working with vendors on accessibility and equity.
Opportunity to learn surveys were completed by multiple states. It would be helpful if this information was compiled in a multi-state summary. We could learn how states are using the data.
Incorporate UDL considerations in assessment item development guidance resources.
Resources that address how to place the most marginalized students front and center (rather than as an exception) in planning efforts.
Resources on how to engage and motivate students, particularly students with the most significant support needs.
Discussion resource on changing assessment formats and score reporting so they better address the needs of all students, including the most marginalized.

Whole Group Discussion
Participants were moved to the main virtual room following the second breakout session. The moderator, Sandra Warren, invited participants to share key takeaways from their discussions with the whole group and thanked participants for sharing their thoughts and ideas.

Debrief and Next Steps

Sheryl Lazarus (Director, NCEO) summarized the day and thanked everyone for their participation. She noted that NCEO will produce a report that contains the forums proceedings. It will be made available to all on the NCEO website (www.nceo.info). The meeting closed with Kathleen Airhart (Program Director for Special Education Outcomes/ASES Collaborative Advisor, CCSSO) thanking attendees for their interest and participation.

Endnotes

¹ The Aspen Education and Society Program and the Council of Chief State School Officers (2017). Leading for equity: Opportunities for state education chiefs.
https://ccsso.org/sites/default/files/2018-01/Leading%20for%20Equity_011618.pdf

² White, K. P. (2013). Person-role-system framework briefing note. Annie E. Casey Foundation.
https://assets.aecf.org/m/blogdoc/PersonRoleSystemFramework-2013.pdf

Forum on Education for Each and Every Student: Perspectives on Universal Design in 2021