Assistive Technology
Interoperability with Online
Assessment Platforms and Other
Technology Products and Services:
Making it All Work!

A publication of:

NATIONAL CENTER ON EDUCATIONAL OUTCOMES

The Center is supported through a Cooperative Agreement (#H326G210002) with the Research to Practice Division, Office of Special Education Programs, U.S. Department of Education. The Center is affiliated with the Institute on Community Integration at the College of Education and Human Development, University of Minnesota. Consistent with EDGAR §75.62, the contents of this report were developed under the Cooperative Agreement from the U.S. Department of Education, but does not necessarily represent the policy or opinions of the U.S. Department of Education or Offices within it. Readers should not assume endorsement by the federal government.  

Project Officer: David Egnor
All rights reserved.  

Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Airhart, K., Strunk, K., Hinkle, A. R., Thurlow, M. L., & Lazarus S. S. (2022). Assistive technology interoperability with online assessment platforms and other technology products and services: Making it all work! National Center on Educational Outcomes.

Background

Approximately 67 individuals representing staff from state departments of education, assessment vendors, and technology companies participated in a forum on June 29, 2022, in Atlanta, Georgia to discuss issues surrounding the use of personal student assistive technology (AT) and online assessment participation. The forum was a post-session to the Council of Chief State School Officers’ (CCSSO) National Conference on Student Assessment (NCSA) and was a collaboration of the Assessment, Standards, and Education for Students with Disabilities (ASES) Collaborative and the National Center on Educational Outcomes (NCEO).

A specific goal of the forum was to gather representatives from state departments of education, assessment vendors, and technology companies to discuss challenges students face when using their personal assistive technology supports to participate in online assessments and to identify possible solutions.

Purpose

The purpose of the forum was for participants to come together to better understand how the technologies and platforms interact, the challenges for students and educators, and how everyone can work together to make improvements and find solutions. This topic grew out of numerous conversations during which states indicated that they were challenged by the interoperability of students’ personal assistive technology supports with online assessment platforms and varying technology systems and products.

Among the questions addressed in the forum were ones related to how states, assessment vendors, and technology companies could work together to improve online assessment experiences for students with disabilities who use assistive technology supports on a daily basis, including during instruction. These included: (a) What types of student personal assistive technology devices and supports are creating challenges? (b) What things are working successfully for these students? (c) What interoperability issues need to be addressed? (d) How can states, assessment vendors, and technology providers work together to improve online assessment participation for students with disabilities who use assistive technology?

The forum started with a presentation from CCSSO that provided an overview of current state technology and data emphases, and the challenges with interoperability states may be addressing. This was followed by presentations from Apple, Pearson, and Texthelp that provided information about their role with states on the provision of their services, common themes of need, and their aspirations for supporting states in the future. (A Google representative was unable to present because of illness.) Finally, the perspectives of two states about their experiences and challenges when helping students with assistive technology supports during online assessment were shared. Following these presentations, forum participants broke into groups, first to discuss the issues, and then to identify needed resources. Participants self-selected one of four groups: (a) learners with high incidence disabilities; (b) learners with sensory disabilities; (c) learners with the most significant cognitive disabilities; and (d) English learners, including English learners with disabilities.

Each of these groups addressed the following questions:

1. What is the specific type of assistive technology students are using that may create challenges with online assessment participation?
2. What is working (the successes) for these students?
3. In a perfect world, what would interoperability look like for these students?
4. In order to have the ideal system, what would states, assessment vendors, and technology providers need to do differently?

Group discussions were rich and engaging. The agenda was as follows:

• Welcome (Kathleen Airhart, CCSSO, and Sheryl Lazarus, NCEO)
• Overview of the State Issues/Solutions for Interoperability (Dean Folkers)
• Technology Company Perspectives
  • Apple Inc. (Patrick Purcell)
  • Pearson (Jan McSorley)
  • Texthelp (Ruth Ziolkowski)
  • ETS ((Mark Hakkinnen)
• State Perspectives
  • Washington State Perspective (Toni Wheeler)
  • Utah State Perspective (Tracy Gooley)
• Breakout Discussions
• Report Out, Next Steps, and Evaluation (Sheryl Lazarus)

Structure of this Report

This report summarizes both the introductory information provided to forum participants and the facilitated forum discussions that followed. Summaries of the presentations were developed from notes taken during the presentations and from the presenters’ slides (and reviewed by presenters). Summaries of the facilitated discussions were developed from notes taken by notetakers.

Participants were encouraged to comment and discuss freely during breakout discussions, with assurances that no individual’s name, nor any state or organization would be attached to comments made. Complete anonymity of statements was ensured. This led to frank and open conversations.

Forum Introduction

Kathleen Airhart (CCSSO ASES advisor) and Sheryl Lazarus (NCEO director) provided an overview of the forum and recognized the hosts—CCSSO and NCEO. They noted that states have expressed challenges with the interoperability of students’ personal assistive technology devices, online assessment platforms, and technology company services during online assessment administrations. States requested a forum to gather representatives from these organizations to share challenges, learn from each other, and discuss ways to improve student online assessment participation experiences.

Overview of State Issues/Solutions for Interoperability

The first presenter, Dean Folkers, Education Data and Technology Director (CCSSO), leads the Modernization and Future Readiness Initiative with partner AEM. This work is focused on supporting State Education Agencies (SEAs) to modernize their data systems. He explained that data systems need to be interoperable and protect privacy and that this Initiative supports SEAs be better able to acquire and adopt secure solutions that maximize the use of data and technology to inform teaching and learning. A goal is to provide capacity, time, and space for SEAs to innovate and think “big picture.” Support is provided to SEAs to determine where they are and build an individual state implementation plan for where they want to go.

The process includes the facilitation of an Interoperability Assessment that allows existing strengths to be used across an organization and in additional offices. It supports the identification of potential improvement areas and short- and long-term priorities for clear and tangible support. Model components include leadership/vision, governance, technology and infrastructure landscape, procurement, implementation fidelity, and privacy. Prior work and existing inputs were leveraged in the development of the new maturity model and assessment rubric including CoSN’s Interoperability Maturity Model, Project Unicorn’s Student Data Rubric, CCSSO/SST’s Data Maturity Model, and the SLDS Maturity Model.

The first cohort of states in the Initiative are California, Massachusetts, Nebraska, and Iowa. A second cohort of states will continue this work at the end of 2022.

Technology Company Perspectives

Apple Inc. Apple established its first team dedicated to accessibility in 1985. In its presentation, Apple defines accessibility in very broad strokes, and disability as a spectrum, and categorizes its features supporting disability communities into four broad categories: vision, mobility, hearing, and cognitive. They discussed common accessibility features, highlighting some that are used in the classroom for learning today, including VoiceOver, Audio Graph, Spoken Content, AssistiveTouch, Voice Control, and Guided Access. Last, they demonstrated how Apple’s assessment mode automatically locks and configures iPad apps into a secure environment, and highlighted the new multi-app mode feature for Mac as of macOS Monterey, which is an important feature for students who need additional apps for their secure tests. Apple welcomed future collaboration on both classroom and assessment needs for all learners.

Pearson Education and Texthelp. Jan McSorley, Vice President of Accessibility, Psychometrics, and Testing Services at Pearson Education, and Ruth Ziolkowski, Senior Vice President of Public Policy and Partnerships at Texthelp, began their presentation by saying that fair and equitable assessments also need to be reliable, interoperable, and secure. A student should be able to use familiar technology in a fair and equitable assessment. Considerations for reliability include teacher and district reliability. For example, some states are using a two-computer solution for accessibility where the second computer has the AT tools. While the two-device solution provides access to familiar technology, students are not used to using two computers and there are several reliability and security challenges, such as:

• Did district staff set up the device correctly (e.g., limiting features, ensuring it is in the locked down browser mode)?
• Did district staff adhere to all security measures or may they have inadvertently shared test content with others?
• Was the second device locked down and not shared with others?

McSorley shared a definition of interoperability: the ability of a system or product to work with other systems or products without special effort on the part of the user. She cited the Elementary and Secondary Education Act (ESEA) requirement for a state’s academic assessment system to provide “the appropriate accommodations,” for each student with a disability, “such as interoperability with, and ability to use, assistive technology devices consistent with nationally recognized accessibility standards, that are necessary to measure the academic achievement of the student” (34 CFR §200.6 (b)). She noted that it is a struggle to make some assistive technologies work with secured test platforms, even when the assistive technologies are delivered through browser extensions. This is why Pearson partnered with Assistive Technology vendors to develop extensions that will work directly with test platforms, thereby solving the security conflict on secured assessments.

Ziolkowski shared some of Pearson’s and Texthelp’s current solutions that address interoperability. She said working together as an assistive technology company with the assessment platform company, an interoperable solution for all environments and platforms has been proven possible and is currently working in five states. They have developed a software development kit (SDK) which is a set of code for developers to use in a protected environment (https://www.npmjs.com/package/@donjohnston/cowriter-sdk-web). She said that this solution is interoperable and reliable, and provides many benefits, including:

• Embedded into the test platform and easy to set up;
• Reduces second device variability at the district level;
• Manages features by state through the test publisher;
• Provides both set and controlled features;
• Ensures alignment with accommodations manuals;
• Minimizes need for training and staff development;
• Works across platforms and in multiple states; and
• Represents the student’s work (no external staff involved in the content).

Security benefits include the test content being secured in the test environment, no printing or scribing is needed, and access to assistive technology is offered through the test publisher’s locked browser. Finally, the solution benefits students by requiring no additional learning demands since the tools work in the same way as other environments. It also lowers test anxiety because the student uses familiar assistive technology. Ziolkowski ended by saying within current testing platforms, states need to be educated so they can opt into these solutions. There are also plans to expand to future collaborations.

Educational Testing Service (ETS). Mark Hakkinnen, Director of Accessibility Standards and Inclusive Technology at ETS highlighted three issues: (a) the need to support a wider range of assistive technology, different versions of assistive technology, and a consistent user experience for assistive technology users; (b) the now wider range of technologies influenced by the pandemic (e.g., students bring their own assistive technology devices); and (c) many different software that are constantly updated often causing new problems that had previously solved.

Hakkinen said that ETS believes that students should expect a consistent user experience with that of their peers. For example, issues such as inconsistent text to speech pronunciation reveals the need for a standard for spoken presentation. Hakkinnen noted that a task force that includes ETS, Pearson, Cambium, and Texthelp has formed to work on this issue. The task force would like feedback from forum participants at https://www.w3.org/WAI/APA/task-forces/pronunciation.

State Perspectives

Washington. Toni Wheeler, Alternate Assessment Coordinator at the Washington State Office of Public Instruction, began her presentation, which was titled Washington Challenges, Successes, and Lingering Questions, by sharing some of the challenges the state is experiencing. The shift from desktop versions to cloud-based apps for third-party software has required districts buy new versions unless they continue using older, outdated desktop versions. The state now has a mishmash of devices, operating systems, software, and versions all needing to “play nice” in a secure browser. The state is unable to quality check all possible combinations, along with embedded and non-embedded supports, especially when a student is using non-embedded supports AND embedded supports.

Another challenge is the perception in some districts that if something is not interoperable, it is not allowed. The state is in an awkward position around equity and access when it has little control. Further, the state is worried that naming products that work well within its testing platform will be misinterpreted as an endorsement and might influence district purchasing.

Wheeler noted that the line between education technology and assistive technology is becoming indistinguishable. What is used for instruction and what is used for assessments is no longer so separate. The pandemic put more devices into students’ hands and there are more accessibility features built into devices and operating systems. This leads to more students, with and without disabilities, using accessibility tools during instruction (e.g., text to speech, dictation, word prediction) and wanting to use them during the assessment.

Washington has had some successes. For example, there is a strong collaborative relationship with the state’s K–12 assistive technology center. The brainstorming and communication of creative solutions is positive as is the state push for more supports to be embedded in the test delivery platform. Another strength is Washington’s Digital Equity and Inclusion Grant available to districts that apply for funding. The funding from this grant is to be used for technology. There is a shift from “what a district wants to get” to “how can all the technology work together?” The grant has two parts. Part 1 involves developing a process whereby educational service districts (ESDs) must work with local school districts to consult on technology needs, assist in technology procurement (taking into consideration needs, in terms of accessibility) and offer training to local school staff so that they can, in turn, provide “digital navigation services to students and their families.” In Part 2 the state assists local schools in achieving “a universal 1:1 student to learning device ratio.” Part of the requirements of the grant is planning for the special technology needs of students with disabilities and English learners for adaptive and assistive technologies.

Wheeler concluded with three lingering questions:

1. Is it better to embed more accessibility tools into the testing platform OR rethink secure browsers to allow more third-party products into the test setting?
2. Is there a tipping point with third-party software/built-ins where it begins to undermine the concept of standardization in a broad sense?
3. Do we need to rethink what we are standardizing to—test administration or instruction?

Utah. Tracy Gooley, Special Education Assessment Specialist at the State Board of Education provided the Utah state perspective during her presentation, Assistive Technology Interoperability with Online Assessment Platforms in Utah. She noted that over time, Utah moved from paper tests to fixed-form multiple-choice computer-delivered criterion-referenced tests, and then in 2014, to computer-adaptive assessments that included technology-based questions. As Utah’s assessments evolved, changes occurred as there were shifts from third party devices to cloud-based extensions to built-in access. Assistive technology also evolved from human based accommodations to third party software such as Dragon NaturallySpeaking™, JAWS®, and eye gaze technology. In 2013, Utah saw the arrival of cloud-based solutions and extension supports such as Read&Write for Google Chrome™, dictation in Microsoft Word and Google Docs™, and Co:Writer®. Built-in technologies were offered as early as 2012, evolved to Chromebooks in 2017, and then Windows in 2017-2018. The devices and technologies most commonly used currently in Utah include Chromebooks (has kiosk mode), Windows, iPad, and MacOS.

Utah has encountered a number of challenges related to operability, including (a) a shift from desktop third-party software to cloud-based apps and built-in accessibility, (b) kiosk mode in Chromebooks, and (c) iOS VoiceOver as a screen reader. There is concern when individualized education program (IEP) teams often want to make decisions based on what technologies will work with the test rather than making decisions about what is best for the student’s instruction. Gooley noted it is challenging to know the right people to talk to and ask for help. Is it vendors, application creators, or others?

Some of Utah’s successes include having open dialogue with vendors about assistive technology needs which led to vendors building additional accessibility features directly into their platforms. They also use “work-arounds,” such as using a document camera to blow up the computer screen to use with a secure browser and having students use a separate device with their assistive technology.

Gooley described the current landscape for technology use in schools: (a) most students have computers they use for everyday instruction in the classroom and at home; (b) assistive technology that was used as an accommodation is now becoming a universal design for learning (UDL) tool in all classrooms and society (e.g., entire schools using Read&Write for Google Chrome™ and students talking into their smart phones to send text messages).

Gooley concluded with three questions for participants to consider:

1. Do we need to rethink “security” when it comes to assistive technology?
2. Does the construct of what is being tested need to be revisited?
3. Should we just have vendors embed features into their platforms?

Session Structure and Outcomes

Following the forum presentations, meeting participants divided into discussion groups of their choice (i.e., learners with high incidence disabilities; learners with sensory disabilities; English learners, including English learners with disabilities; learners with the most significant cognitive disabilities). Discussions focused on four questions, although not all were addressed in each group:

1. What type of assistive technology are you referencing?
2. What is working (the success) for these students?
3. In a perfect world, what would interoperability look like for these students?
4. In order to have the ideal system, what would states, assessment vendors, and technology providers need to do differently?

Learners with High Incidence Disabilities

Types of Assistive Technology. Several specific types of assistive technology were mentioned, sometimes with issues identified.

• Speech to text
• Text to speech (challenge is that some teachers think it must be used always or never, even though it can be used only at certain times; also, variability in voices can be problematic)
• Google voice typing (although this does not transfer to the test platform)
• Eye gaze (most challenges are with Chromebooks and iPads)

What is Working. Some assistive technology supports seemed to work well:

• Eye gaze
• Built in cloud features such as dictation
• Using Chromebook with Kiosk mode
• Online IEP system integrated with testing

Perfect World Interoperability. Several comments were made about what interoperability should look like:

• Choices for assistive technology available based on complexity and variability of student needs
• Dictation for everyone
• IEP links directly to the test platform in order to provide better support, accessibility, and accommodation making for specific tests
• Consistent use of universal design across instruction and assessment

What Needs to be Different.A few comments identified what needs to change:

• Consistency across platforms is needed
• Better communication across partners
• Better understanding of the challenges schools are facing by vendors and assistive technology developers
• More personalization and customization

Learners with Sensory Disabilities

Types of Assistive Technology. A few specific types of assistive technology were mentioned, along with some challenges.

For students who have visual impairments or are blind:

• Refreshable braille devices, although they are expensive and the software is difficult to learn
• JAWS®™
• Braille notetaker
• Chunking of the assessment for braille readers
• Screen readers, although these are challenging in math

For students who have hearing impairments or are deaf:

• Sign language interpreters (A challenge is that it can be difficult to know whether interpreters are qualified, and it can be difficult to train interpreters on how to appropriately sign an assessment.)
• Embedded videos of American Sign Language (ASL) (A challenge is that math videos are often not well done.)
• Clear masks to allow for lip reading

What is Working. The assistive technology mentioned in response to the question about the types of assistive technology being used (e.g., refreshable braille devices for students who are blind; embedded video of ASL for students who are deaf) were generally considered to be working. Still, participants often qualified their answer (e.g., expensive, training needed).

Perfect World Interoperability. Several comments were made about what interoperability should look like:

• Braille notetakers, screen readers, and refreshable braille would seamlessly work together
• District personnel would think ahead and ask early for approval of unique assistive technology accommodations requests (and would understand the reasons behind state policies)
• Consistency in available assistive technology supports across assessments at both state and local levels
• IEP teams would understand assistive technology, and realize that it is not always a machine or software

What Needs to be Different. A few comments identified what needs to change:

• Vendors need to work together and listen to states’ needs
• Systems should not be updated by venders during testing windows
• Districts need to understand what is available and that a student can use allowed assistive technology supports during assessments
• Recognition of assistive technology costs, as well as the need for training
• Consistency across vendors in embedded assistive technology and platform design

English Learners, Including English Learners with Disabilities

Types of Assistive Technology. Several specific assistive technologies were mentioned, sometimes with issues identified:

• Electronic dictionaries (bilingual or English)
• Translate any language at any time at a reasonable cost (This needs to also be available in an audio format because some students may have difficulty interacting with print)
• Google Translate™ (When there is a lack of translators; it is used in the classroom all day long, but a challenge is that it cannot be put in the “locked kiosk mode” for assessments.)
• Printed bilingual dictionaries for less common languages (A challenge is that some dictionaries provide too much information.)
• Student writing in native language, which is then translated for scoring
• Keyboard stickers for various languages and alphabets (e.g., Arabic, Cyrillic)

What is Working.Several approaches to assistive technology were mentioned:

• A toggle function so students can switch between English and Spanish; at least one vendor provides this and some like this better than stacked translation
• Speech to text in Spanish
• Built-in dictionary in Spanish (This could be in other languages, but they are not available yet.)
• Hover over words then translate into Spanish (Although this is not equitable if does not translate into other languages.)

Perfect World Interoperability.Several comments addressed what things should be like:

• Low cost or no cost
• Lots of languages (more than in-demand languages and dialects)
• Easier approaches (e.g., hover over vs. stacked dictionaries)
• Visual component in dictionary
• One-to-one correspondence with classroom assistive technology (e.g., same names, same tools)
• When working on assessment requests for proposals (RFPs), writers need to include language about being responsive to emerging technologies (and how they will deploy them within a year) at no additional cost, with punitive damages if it does not happen

What Needs to be Different. Several comments addressed what needs to change:

• Information needs to be collected on assistive technology being used by English learners with disabilities
• Rethink test security (How secure do tests need to be?)
• Need for multilingual assistive technology
• Vendors should not be able to say no to state requests; instead, they should engage in problem solving
• Include students in thinking about assistive technology innovations
• Instructional Management System (IMS) Global Learning Consortium publishes interoperability standards; state memberships focus on common language
• Assistive technology supports embedded in Accessible Portable Item Protocol (APIP) and Question and Test Interoperability® (QTI®) needs to be expanded
• More effort needs to be made to keep pace with changing technology and student needs

Learners With the Most Significant Cognitive Disabilities

Types of Assistive Technology. Discussion of assistive technology for these students was embedded in a discussion of challenges:

• Eye gaze, although there are many challenges to getting this to work with the types of items in alternate assessments (e.g., selecting among cards) and it requires two separate systems to work together
• Switch systems, although most systems are two-switch systems; one approach that seems innovative is a keyboard with a switch that operates like a mouse
• Electronic scribe
• Word prediction
• Human reader rather than text-to-speech because the latter often has unnatural cadence and unusual pronunciations; also, refreshable braille and even braille are difficult for many students with the most significant cognitive disabilities

What is Working. Most current approaches to assistive technology have numerous challenges for students with the most cognitive disabilities. Still two positives were mentioned:

• Computer adaptive testing
• Alternate assessment, in general, is more accessible than the general assessment

Perfect World Interoperability. A few comments addressed what would exist in a perfect world for students with the most significant cognitive disabilities:

• Correspondence between assistive technology used in instruction and in assessment
• Trained and knowledgeable teachers who understand how to use assistive technology
• An IEP form that requires assistive technology to be considered for each student
• Teacher preparation programs for these students that address assistive technology

Conclusion

The meeting closed with concluding remarks by Sheryl Lazarus (NCEO). She thanked participants for their thoughtful discussions and confirmed that a report with the forum proceedings would be available in the fall of 2022.

Top of page