NCEO Report 423

A Summary of the Research on the Effects of K–12 Test Accommodations: 2018

Christopher M. Rogers, Sheryl S. Lazarus, and Martha L. Thurlow

February 2021

All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Rogers, C. M., Lazarus, S. S., & Thurlow, M. L. (2021). A summary of the research on the effects of K–12 test accommodations: 2018 (NCEO Report 423). National Center on Educational Outcomes.

Table of Contents

 

Executive Summary

The use of accommodations during assessments continues to be important for students with disabilities and for states as they establish and revise their accommodations policies. This importance is reflected in continued research to investigate the effects of accommodations. Key issues under continued investigation include how accommodations affect test scores, how educators and students perceive accommodations, and how accommodations are selected and implemented.

The purpose of this report is to provide an update on the state of the research on testing accommodations. Previous reports by the National Center on Educational Outcomes (NCEO) have covered research published since 1999. In this report, we summarize the research published in 2018. During 2018, 11 research studies were published on the topic of testing accommodations in the U.S. elementary and secondary education system.

Purpose of research: Over 80 percent of the research published in 2018 was to evaluate the effects on test scores when K–12 assessments were administered with accommodations. The next most frequent purpose was to report on perceptions and preferences about accommodations use. The majority of studies (about 82%) addressed multiple purposes, and about half of the studies investigated both the performance impact and test-taker perceptions of accommodations.

Research design: All studies reported primary data collection on the part of the researchers, rather than drawing only on existing data sets. Experimental or quasi-experimental designs were used in over 60 percent of the studies. Researchers also applied a variety of other quantitative and qualitative methodologies, including survey data analyses and interview protocols.

Types of assessments, content areas: A wide variety of instrument types were used. Over two-fifths of the studies used academic content items drawn from specific sources outside of the study authors’ work, and almost one-fifth of studies used state criterion-referenced tests. Over four-fifths of the studies used non-academic protocols or surveys developed by the study authors. A few studies (27%) used norm-referenced measures. Nearly all of the studies (82%) used multiple types of data. Mathematics and reading were the most common K–12 content areas addressed in the research published in 2018. Other content areas included science and English language arts. One-fifth of all studies addressed more than one content area.

Participants: Participants were most frequently students, spanning a range of grades throughout K–12 education, including the elementary, middle, and high school levels. Over one-third of studies had participants in more than one school level. Some studies included educators or students’ parents as participants. Studies varied in the number of participants; some studies included fewer than 10 participants, while over half of the studies included more than 30 participants, and one study had nearly one thousand.

Disability categories: Visual impairment including blindness was the most common disability category of participants in the research published in 2018, accounting for more than one-third of the studies. Learning disabilities comprised the next most commonly studied category. Attention-related disabilities, intellectual disabilities, and hearing impairment including deafness were included in one quarter of the studies altogether.

Accommodations: Presentation accommodations were the most frequently studied category of accommodations. Electronic administration was the most-studied individual accommodation. A relatively large proportion of the studies published in 2018 reported on accommodations (e.g., calculator, sign language administration) that were investigated in only one study each.

Findings: Nine studies analyzed the effect of accommodations. Of these nine, the accommodation studied most often was electronic administration pertaining to how test-takers are presented assessments as well as how they respond to assessments, and whether additional equipment is required for assessment participation. Other findings on the effects of accommodations were reported by only one study each. Seven studies provided findings on effects of accommodations on math assessments, three studies presented findings on effects for reading assessments, and two studies analyzed the effects of accommodations in more than one content area. Some studies provided comparisons of different versions of accommodations rather than focusing on the effects of a single accommodation. In many studies, accommodations benefited at least some students with disabilities, yet had no effect on the performance of other students with disabilities. No studies indicated a negative effect of accommodations for students with disabilities.

Almost two-thirds of the studies provided findings about perceptions of accommodations. Over one quarter of K–12 accommodations studies published in 2018 provided findings about accommodations use patterns by students or implementation practices by educators. Many studies provided insights about students’ general impressions about accommodations as well as their preferences among accommodations. Most students with disabilities had positive perceptions of accommodations. Relatively few findings were available on educators’ and parents’ perceptions about accommodations.

Table of Contents

 

Overview

Access to assessments for students with disabilities is supported through the use of accommodations. With accommodations, students with disabilities, including English learners with disabilities, are better able to show their academic knowledge and skills. Accommodations also enable these students to participate in state assessments, as required by the Individuals with Disabilities Education Act (IDEA) of 2004 and by the 2015 reauthorization of the Elementary and Secondary Education Act (ESEA). Accommodations are changes in materials and procedures that do not compromise the validity of assessment results and interpretations of those results. Evidence is needed to ensure that validity is not compromised. It is also important to examine perceptions of accommodations and implementation issues because these influence whether accommodations are used appropriately. Research conducted on accommodations can provide states with information useful for policy on accommodations.

To synthesize accommodations research efforts completed across the years, the National Center on Educational Outcomes (NCEO) has published a series of reports on accommodations research. The time periods included 1999–2001 (Thompson et al., 2002), 2002–2004 (Johnstone et al., 2006), 2005–2006 (Zenisky & Sireci, 2007), 2007–2008 (Cormier et al., 2010), 2009–2010 (Rogers et al., 2012), 2011–2012 (Rogers et al., 2014), 2013–2014 (Rogers et al., 2016), 2015–2016 (Rogers et al., 2019), and 2017 (Rogers et al., 2020). The report summarizing the 2017 empirical studies narrowed the focus to K–12 research in the United States context.

The purpose of this report is to present a synthesis of the research on test accommodations for U.S. elementary and secondary students (K–12) published in 2018. The academic literature described here encompasses empirical studies of performance comparability, investigations into accommodations use, implementation practices, and perceptions of the effectiveness of accommodations. As a whole, the current research body offers a broad view and a deep examination of issues pertaining to assessment accommodations. Reporting the findings of recent research studies was the collective goal of these analyses.

Review Process

Similar to the process used in NCEO’s past accommodations research syntheses, a number of sources were accessed to complete the review of the K–12 accommodations research published in 2018. Specifically, five research databases were consulted: Educational Resources Information Center (ERIC), PsycINFO, Academic Search Premier, Digital Dissertations, and Educational Abstracts. To help confirm the thoroughness of our searches, we used the web search engine Google Scholar to locate additional research. In addition, a hand-search of at least 50 journals was completed in efforts to ensure that no qualifying study was missed. A list of hand-searched journals is available on the NCEO website (https://nceo.info/Resources/bibliographies/accommodations/methods-for-identifying).

Online archives of several organizations also were searched for relevant publications. These organizations included the following: Behavioral Research and Teaching (BRT) at the University of Oregon (https://www.brtprojects.org/publications/); the College Board Research Library (http://research.collegeboard.org); the National Center for Research on Evaluation, Standards, and Student Testing (CRESST; http://cresst.org/education/); and the Wisconsin Center for Educational Research (WCER; https://www.wcer.wisc.edu/publications).

The initial search was completed in December, 2018. A second search was completed in April 2019 to ensure that all articles published in 2018 were found and included in this review. Within each of these research databases and publications archives, we used a sequence of search terms. Terms searched for this review were:

Many of these search terms were used as delimiters when searches yielded large pools of documents found to be irrelevant to the searches.

The research documents from these searches were then considered for inclusion in this review using several criteria.

  1. This analysis included only research published or defended (in doctoral dissertations) in 2018.
  2. The scope of the research was limited to investigations of accommodations for regular assessments; hence, articles specific to accommodations for alternate assessments, accommodations for instruction or learning, and universal design in general were not part of this review.
  3. Research involving English learners was included only if the target population was English learners with disabilities.
  4. Presentations from professional conferences were not searched or included in this review, based on NCEO’s criterion to include only research that would be accessible to readers and had gone through the level of peer review typically required for publication in professional journals or through a doctoral committee review. (This criterion was implemented for the first time during the 2007–2008 review.)
  5. To be included in the online bibliography and summarized in this report, studies needed to involve (a) experimental manipulation of an accommodation; (b) investigation of the comparability of test scores across accommodated and non-accommodated conditions, or across more than one accommodated condition; or (c) examination of survey results or interview data sets about students’ or teachers’ knowledge or perceptions of accommodations.
  6. This report was focused on research on students in United States schools; consequently, studies with only participants in other national contexts were not included.
  7. The current report includes only research pertaining to the primary and secondary levels of the education system, that is, from Kindergarten through grade 12.
  8. This report does not include literature reviews or meta-analyses (unlike previous NCEO accommodations research reports).

These limitations do not necessarily apply to NCEO’s Accommodations for Students with Disabilities Bibliography, which is an online database (https://nceo.info/Resources/bibliographies/accommodations/bibliography). The Accommodations Bibliography continues to include research in non-U.S. settings. Also, postsecondary accommodations research is included in the Bibliography. Many literature reviews of various kinds also are included in the Bibliography.

To reflect the wide range of accommodations research in the K–12 system that was published in 2018, the studies are examined and summarized for the following features: (a) publication type, (b) purposes of research, (c) research type and data collection source, (d) assessment or data collection focus, (e) characteristics of the independent and dependent variables under study, and (f) comparability of findings between studies in similar domains.

Table of Contents

 

Results

Publication Type

A total of 11 studies was published between January 2018 and December 2018. As shown in Figure 1, nine of the 11 studies were journal articles, and two were dissertations. None were published professional reports released by research organizations or entities (e.g., WCER).

The total number of studies published on accommodations in the K–12 educational context in 2018 (N=11) decreased from 2017, when it was 14, yet was similar to previous years. In NCEO’s previous report (Rogers et al., 2020), we applied our more stringent inclusion criteria for analyzing the research literature. The numbers of published K–12 studies were nine in 2015 and nine in 2016. The numbers of journal articles increased, from four in 2015, to seven in 2016, to seven in 2017, to nine in 2018. The largest variability in type was for dissertations, with five in 2015, two in 2016, seven in 2017, and two in 2018. The absence of reports from research organizations has continued, with zero in 2015, 2016, 2017, and 2018. This (2018) review included nine journal articles from seven different journals, including three articles from one academic journal. In previous years’ data, there were seven journal articles from seven different journals in 2017 (Rogers et al., 2020), seven articles from seven different journals in 2016, and four articles from four different journals in 2015 (Rogers et al., 2019). Appendix A presents information about the publication type of each study published in 2018.

Figure 1. Percentage of K–12 Accommodations Studies in 2018 by Publication Type

Figure 1 Pie Chart

Purposes of the Research

Several purposes were identified in the K–12 accommodations research published in 2018. Table 1 shows the primary focuses of the 11 studies included in this review. Two studies each listed a single purpose (see Appendix B). The majority of studies sought to accomplish multiple purposes. In these cases, we identified the “primary purpose” based on the title of the work or the first-mentioned purpose in the text.

Table 1. Primary Purpose of K–12 Studies in 2018

Purpose Number
of Studies
Percent
of Studies
Discern effects on assessment scores 8 73%
   only students with disabilities (6 studies; 55% of studies)
   only students without disabilities (2 studies; 18% of studies)
   both students with and without disabilities (0 studies; 0% of studies)
Report on implementation practices and accommodations use 2 18%
Study/compare perceptions and preferences about use 1 9%
Discuss issues 0 0%
Review or summarize research on test accommodations 0 0%
Compare test items 0 0%
Evaluate test structure 0 0%
Investigate test validity 0 0%

The most frequent primary purpose for research published during 2018 was discerning the effects of accommodations through score comparisons during test performance. Two studies’ primary focus was on accommodations implementation practices or accommodations use. Only one study focused primarily on the perceptions of accommodations.

Table 2 shows the multiple purposes investigated by nine of the 11 studies, as well as the singular purpose of two studies. Although primary purposes were limited to three (effects, implementation or use, and perceptions), many studies had more than one purpose. Nine studies analyze, either as a primary or other purpose, the effects of accommodations through comparing performance data for students only with disabilities (n=7) and for students only without disabilities (n=2). The next most widely studied purpose (n=7) was examinations of the perceptions of students, teachers, or students’ parents on accommodations. Three studies described implementation practices or incidence of use during assessment. Two studies each summarized relevant accommodations research or discussed accommodations-related issues.

Appendix B presents the studies’ many purposes. Nearly all of the studies (n=9) addressed two or more purposes. Six studies (Bone & Bouck, 2018; Bouck et al., 2018; Ha & Fang, 2018; Hansen et al., 2018; Kong et al., 2018; McLaughlin & Kamei-Hannan, 2018) had two purposes, with almost all (except McLaughlin & Kamei-Hannan) including analysis of effects of accommodations and examining participants’ perceptions. Two studies (Cobb, 2018; Pangatungan, 2018) investigated three purposes. Beal and Rosenblum (2018) addressed four purposes, including effects of accommodations, perceptions and preferences between accommodations, implementation and use, and discussing accommodations-related issues.

Table 2. All Purposes of K–12 Studies in 2018

Purpose Number
of Studies
Percent
of Studies
Discern effects on assessment scores 9 82%
   only students with disabilities (7 studies; 64% of studies)
   only students without disabilities (2 studies; 18% of studies)
   both students with and without disabilities (0 studies; 0% of studies)
Report on implementation practices and accommodations use 3 27%
Study/compare perceptions and preferences about use 7 64%
Discuss issues 2 18%
Review or summarize research on test accommodations 2 18%
Compare test items 0 0%
Evaluate test structure 0 0%
Investigate test validity 0 0%

Note. Of the 11 studies reviewed for this report, 9 reported addressing two or more purposes. Thus, the number of studies totals more than 11, and more than 100%.

Research Type and Data Collection Source

Quasi-experimental design was the most frequent type of accommodations research published in 2018, comprising nearly one-half of the 11 K–12 studies. As shown in Table 3, the researchers for all quasi-experimental studies, and in fact, all 11 studies, gathered the data themselves. One of the studies (Cobb, 2018) also included secondary data in an analysis of the high school’s annually-collected state assessment performance scores. Three studies were descriptive quantitative designs, and two studies reported using experimental designs. Only one study used primarily or exclusively qualitative data. No studies published in 2018 used correlational or longitudinal designs, so those designs are not included in Table 3.

Table 3. Research Type and Data Collection Source for K–12 Studies in 2018

Research Design Data Collection Source Total
  Primary Secondary  
Quasi-experimental 5 0 5
Descriptive quantitative 3 01 3
Experimental 2 0 2
Descriptive qualitative 1 0 1
Totals 11 0 11

1 One study collected primary data, yet also examined extant state assessment data.

We observed a difference in the number of studies published in 2018 that used primary data collection sources when compared to those that used secondary data collection sources. All 11 studies used primary sources and none used only secondary sources. In studies published in 2017, we found that over 20 percent of the studies employed secondary data sources. (Appendix A presents research designs and data collection sources for individual studies.)

Data Collection Methods and Instruments

Research published in 2018 used the methods shown in Figure 2 to collect study data. Seventy-three percent of the studies (n=8) used performance data acquired through academic content testing. In some of the cases (e.g., Hansen et al., 2018), tests were administered as part of the study, while in one case (Cobb, 2018), an extant data source was used. Observations comprised a large proportion (55%) of studies’ data, most commonly task or test completion times (n=4, 36%). In contrast, observation data comprised about 14 percent of the 2017 studies (Rogers et al., 2020). Other frequent data sources in 2018 K–12 studies include surveys (n=6, 55%) and interviews (n=4, 36%). Studies did not report academic grade records as relevant data. Nine studies (82%) reported using more than one method or tool to gather data. See Appendix A for additional details about each study’s data collection methods.

Figure 2. Data Collection Methods Used in K–12 Studies in 2018

Figure 2 Bar Chart

Note. Of the 11 studies reviewed for this report, 5 reported using two data collection methods and 4 reported using three data collection methods. Thus, the number of methods in this figure totals more than 11.

All of the studies published in 2018 used some type of data collection instrument (see Table 4). The instruments were placed into seven categories based on the sources of the instruments:

In 10 studies, non-academic protocols developed by the authors of the studies were used. This was the most commonly-used type of instruments. Examples included questionnaires with rating scales of social validity checks on the students’ testing experiences (Bone & Bouck, 2018), interview protocols for uncovering students’ perceptions about ASL clarity as well as preferences between signer versions (Hansen et al., 2017), and observation protocols for measuring reading speed and accuracy (McLaughlin & Kamei-Hannan, 2018).

Surveys or academic tests developed by researchers or other education professionals using sources outside of the study were employed in seven studies. An example of a survey in the studies in 2018 was adapted by the researchers (Ha & Fang, 2018) from the USE Questionnaire (Lund, 2001) containing elements of usability: usefulness, satisfaction, and ease of use. An example of an academic test that was created drew mathematics practice items for preparing for a standardized math assessment, linked to academic standards expected of a student at the middle school level or early high school level (Hansen et al., 2018).

Table 4. Data Collection Instrument Sources for K–12 Studies in 2018

Instrument Source/s Number
of Studies
Percent
of Studies
Non-academic surveys, interview protocols, and observation protocols
developed by study author/s
10 91%
Surveys or academic tests developed by professionals or researchers
using sources outside of current study
7 64%
State criterion-referenced assessments 2 18%
Norm-referenced academic achievement measures 2 18%
Norm-referenced cognitive ability measures 2 18%
Criterion-referenced (non-state) academic achievement measures 0 0%
Othera 3 27%

Note. Nine studies (82%) used instruments from more than one source; therefore, numbers total more than the 11 studies represented, and percentages total more than 100.

a Other: see Appendix C, Table C-1 for specific information in Bara et al., 2018; Bouck et al., 2018; and Kelly et al., 2018.

Table 4. Data Collection Instrument Sources for K–12 Studies in 2018 (continued)

KEY

Surveys sets of items of an attitudinal or self-report nature
Tests either course- or classroom-based
Assessments statewide or large-scale assessments
Protocols sets of questions, usually presented in an interview format
Measures norm-referenced academic achievement or cognitive ability instruments

State criterion-referenced assessments included the extant data set from the 2017 Smarter Balanced Assessment Consortium (SBAC) English language arts and mathematics assessments administered in California (Cobb, 2018), as well as 40 publicly-available items from already-completed test booklets for the 2013 Texas Assessment of Academic Readiness grade 3 reading achievement test (Kelly et al., 2018). Two norm-referenced academic achievement measures were used for pre-screening or independent checking of performance, including the Kaufman Test of Educational Achievement, Second Edition (KTEA™-II; Kaufman & Kaufman, 2004) and the Johns Basic Reading Inventory (Johns, 2012). Three norm-referenced cognitive ability measures were each used in one study: the Key Math-3 Diagnostic Assessment (KeyMath-3; Connolly, 2007), the Wechsler Intelligence Scale for Children, Fourth Edition (WISC, 4th ed.; Wechsler, 2004), and the Purdue Spatial Visualization Test: Revised (PSVT-R; Yoon, 2011). Criterion-referenced academic achievement measures were not utilized in the 2018 studies. Over 80 percent of all studies (n=9) used instrumentation of more than one kind. We present a complete listing of instruments used in each of the studies in Table C-1 in Appendix C, including the related studies or other sources for these instruments, when available.

Content Area Assessed

Nine studies published during 2018 focused on accommodations used in specific academic content areas (see Appendix C, Table C-2, for additional details about the content areas). As shown in Table 5, mathematics was the most commonly studied content area. Table 5 was constructed, in part, by applying the same criteria used in the current report retroactively to the data from the 2015–2016 report (Rogers et al., 2019), as well as drawing data from the 2017 report (Rogers et al., 2020). In three of the four years of our accommodations research literature reviews (all except 2016), reading and mathematics were the most common content areas for accommodations research, yet have varied in terms of which of the two was the most common content area in any particular year.

Cumulatively, science has tended to be the third most frequent content area, with 9 total studies across the four years of research reviews. In 2018, about one-fifth of studies examined accommodations impact data for more than one content area. The inclusion of multiple content area analyses varied in frequency across the four years, from none in 2016 to four (36%) in 2017. The remaining academic content areas—writing, “other language arts,” and social studies—comprised similar proportions of the research across the four years of studies.

Table 5. Academic Content Area Assessed in K–12 Studies across Three Reports

Content Area Assessed 2015a 2016 2017b 2018c
Mathematics 3 (33%) 1 (14%) 5 (45%) 7 (64%)
Reading 3 (33%) 1 (14%) 6 (55%) 5 (46%)
Writing 2 (22%) 1 (14%) 1 (9%) 0 (0%)
Other language artsd 1 (11%) 0 (0%) 2 (18%) 1 (9%)
Science 1 (11%) 4 (57%) 3 (27%) 1 (9%)
Social studies 1 (11%) 0 (0%) 0 (0%) 0 (0%)
Multiple contente 2 (22%) 0 (0%) 4 (36%) 2 (18%)
Total (of Relevant Studies) 9 7 11 11

a Studies in 2015 included studies that addressed two content areas.

b Studies in 2017 included studies that addressed more than one content area (i.e., two content areas, three content areas).

c Studies in 2018 included studies that addressed more than one content area (i.e., two content areas, three content areas).

d Detailed descriptions of what constituted “Other Language Arts” for the 2018 studies can be found in Appendix C, Table C-2.

e Because some studies investigated effects in more than one content area, the percentages total more than 100.

Research Participants

The studies in this examination of accommodations research published in 2018 included participants in several roles (see Figure 3 and Appendix D). A majority of the studies included only students (n=8, 73%). Only educators participated in one study (Cobb, 2018). The remaining studies had combinations of participant populations. Students with visual impairments and teachers participated in the Beal and Rosenblum (2018) study that examined effects of, and perspectives on use and practices related to, electronic tablet-based assessments. Pangatungan (2018) sought the viewpoints of the parents and teachers of students with and without learning disabilities on the students’ use of oral delivery for supporting academic performance and behavior.

Table 6 details the size and composition of the participant groups in the K–12 research studies published during 2018. For additional detail by study, see Appendix D. The size of the participant groups varied from three participants (Bara et al., 2018; Bouck et al., 2018; McLaughlin & Kamei-Hannan, 2018) to 964 (Kong et al., 2018). The largest single set of studies published in 2018 involved only students with disabilities (5 studies); in comparison, only two studies (Ha & Fang, 2018; Kong et al., 2018) had participants who were only students without disabilities. One study (Kelly et al., 2018) compared groups of students with and without disabilities, yet no studies directly compared data from an identical number of students from both populations. Two studies engaged only educators (Cobb, 2018) or educators and parents (Pangatungan, 2018) as participants, and one study (Beal & Rosenblum, 2018) had both educators and students as participants.

Figure 3. Types of Research Participants for K–12 Studies in 2018

Figure 3 Bar Chart

Two studies engaged only educators (Cobb, 2018) or educators and parents (Pangatungan, 2018) as participants, and one study (Beal & Rosenblum, 2018) had both educators and students as participants.

Table 6. Participant Sample Sizes and Ratio of K–12 Students with Disabilities in 2018

Number of Research
Participants by Study
Number of Studies by Proportion of Sample
Comprising Individuals with Disabilities
  0–24% 25–49% 50–74% 75–100% Total
1–9     0 0 0 4 4
10–24     1 0 0 0 1
25–49     2 0 0 1 3
50–99     0 0 1 1 2
100–1000     1 0 0 0 1
Total     3 0 1 7

School Level

Students in the elementary, middle, and high school levels participated in nine of the 11 studies published in 2018 (see Table 7; also see Appendix D for students’ specific grade levels when available). Most studies included students in more than one grade; only two studies (Bara et al., 2018; Bone & Bouck, 2018) each examined student data for a single grade level. Four studies involved students from more than one school level: one study (Kelly et al., 2018) had elementary and middle school participants, one study (McLaughlin & Kamei-Hannan, 2018) had middle and high school participants, one study (Hansen et al., 2018) had high school and some postsecondary students, and one study (Beal & Rosenblum, 2018) had participants from all three school levels.

Table 7. School Level of Research Participants for K–12 Studies in 2018

Education Level of Participants in Studies Number of Studies Percent of Studies
Elementary school (K–5) 3 27%
Middle school (6–8) 6 55%
High school (9–12) 4 37%
Not applicable 2 18%

Note. Four studies (37%) had participants in more than one education level; therefore, the numbers total more than the 11 studies represented, and percentages total more than 100.

Disability Categories

The K–12 accommodations research published in 2018 addressed a number of disability categories (see Appendix D for details). As shown in Table 8, two studies included only students without disabilities, and one study included both students with and without disabilities; two studies did not include students in the sample, and one study included both students and non-students in the sample. Of the nine studies including students with disabilities, the most commonly studied student disability category was visual impairment including blindness (n=4). Two studies (Bara et al., 2018; McLaughlin & Kamei-Hannan, 2018) included only participants with visual impairments. One study (Kelly et al., 2018) compared students with visual impairments to students without disabilities. One study (Beal & Rosenblum, 2018) included students with visual impairments and their teachers.

Table 8. Disabilities Reported for Research Participants for K–12 Studies in 2018

Disabilities of Research Participants Number of Studies Percent of Studies
Visual impairment/blindness 4 36%
Learning disabilities 2 18%
Attention problem 1 9%
Hearing impairment/Deafness 1 9%
Intellectual disabilities 1 9%
No disability 3 27%
Not applicable (non-students) 3 27%

Note. Some studies had participants who fit into more than one disability category; therefore, the numbers total
more than the 11 studies represented, and percentages total more than 100.

In addition to visual impairments, one other study included students from only one disability category. Specifically, Hansen and colleagues (2018) gathered data from students with hearing impairments including deafness who used American Sign Language in the educational setting. Two studies (Bone & Bouck, 2018; Bouck et al., 2018) included students from two disability categories. The Bone and Bouck study analyzed data from students with learning disabilities and those with learning disabilities plus attention problems.

Types of Accommodations

The specific types of accommodations included in the research published in 2018 are summarized in Table 9. A complete listing of accommodations examined in each study is provided in Appendix E, Table E-1, and by accommodation type in Tables E-2 through E-6.

Table 9. Accommodations in Reviewed Research in 2018

Accommodations Category Number of Studies
Presentation 7
Equipment/Materials 5
Response 4
Timing/Scheduling 0
Setting 1

Note. Five studies investigated accommodations from more than one category; therefore, the numbers in this table total more than the 11 studies represented.

Presentation was the most commonly investigated accommodation category, in seven studies. Equipment/materials accommodations were addressed in five studies, and response accommodations were examined in four studies. Much less frequently-studied was setting, in two studies. Some studies (n=4) included accommodations from more than one category. Of those, three studies included accommodations from each of three accommodations types, and one study included accommodations from two accommodations types.

The most broadly studied presentation accommodation (see Appendix E, Table E-2) was oral delivery, in three studies (Bara et al., 2018; Hansen et al., 2018; Pangatungan, 2018). In previous NCEO accommodations research reports, we merged the data for the three ways that this accommodation was provided: (a) by a test administrator live and in-person, (b) with a recorded human voice, and (c) as simulated speech via text-to-speech devices or software. Oral delivery presented live and in-person was combined with two-dimensional illustrations and tactile illustrations supports in one study (Bara et al.). Recorded video delivery of American Sign Language (ASL) by a person, and ASL simulated by an avatar through software, were compared in one study (Hansen et al.). One study (Pangatungan) described perceptions of live, in-person, oral delivery and text-to-speech software.

Details on equipment accommodations are presented in Appendix E, Table E-3. Electronic administration was investigated in three studies (Beal & Rosenblum, 2018; Kong et al., 2018; McLaughlin & Kamei-Hannan, 2018). This accommodation encompasses three types of accommodation: (a) how students are presented tests, (b) computer or electronic tablet equipment, and (c) the way students record their test item responses. Two studies (Beal & Rosenblum, McLaughlin & Kamei-Hannan) investigated the effects of being presented tests on paper or electronically: one study addressed effects on reading speed (McLaughlin & Kamei-Hannan), and the other combined large print or braille with either paper or electronic tablet to discern any effects on performance (Beal & Rosenblum). In one study (Kong et al., 2018), researchers explicitly compared administration by computer to administration by electronic tablet, in order to discern whether students’ use of keyboards or touchscreens for recording their responses affected test item response speed.

The equipment/materials accommodation of technological aids was the focus of two studies (Bouck et al., 2018; Ha & Fang, 2018), in addition to electronic administration (Beal & Rosenblum, Kong et al., McLaughlin & Kamei-Hannan).

Details on response accommodations are presented in Appendix E, Table E-4. The response accommodation of calculator was researched in one study (Bone & Bouck, 2018). Three response accommodations studies focused on electronic administration studies.

Setting accommodations were investigated in one study (see Appendix E, Table E-5). Kelly and colleagues (2018) examined individual administration.

No studies published in the 11 2018 studies examined timing or scheduling accommodations. In previous years, a common timing or scheduling accommodation was extended time.

Research Findings

The findings of the studies on accommodations published in 2018 are summarized here according to results of the studies. Appendix F presents details on individual studies. These findings were consistent with the stated purposes and focuses of the studies. The findings included sets of research about specific accommodations, such as tactile graphics. Other studies examined impacts of aggregated sets of accommodations sometimes called “bundles.” We also identify findings on the impact of unique accommodations—those examined in only one study—such as scientific calculators, and touch screen or keyboard response. We report on perceptions of accommodations, including those of student test-takers, educators, and students’ parents. We describe implementation conditions as well as patterns of use of various accommodations. This report also presents findings by academic content areas: math, reading and other language arts, and science.

Impact of Accommodations

Research published in 2018 that examined the effects of accommodations on assessment performance for K–12 students with disabilities totaled 9 studies (see Figure 4; see also Appendix F for details about each of these studies). We report here the effects of three discrete accommodations, including tactile graphics (illustrations or manipulatives), oral delivery—live and in-person, as well as oral delivery with simulated voice via text-to-speech software—and electronic administration. See Appendix F for further details on accommodations with only one associated finding.

Figure 4. Effects of Specific Accommodations for K–12 Studies in 2018

Figure 4 Bar Chart

Note. Five studies each examined the separate impacts of more than one accommodation; therefore, the total exceeds the number of studies represented (9).

The two most investigated accommodations in 2018 were investigated in three studies each: electronic administration (Beal & Rosenblum, 2018; Kong et al., 2018; McLaughlin & Kamei-Hannan, 2018) and tactile graphics (Bara et al., 2018; Bouck et al., 2018; Ha & Fang, 2018). Electronic administration was provided in comparison with standard paper format (Beal & Rosenblum, McLaughlin & Kamei-Hannan) or comparing computer with keyboard responding against electronic tablet with touchscreen responding (Kong et al.). Tactile graphics were presented both as low-tech physical manipulatives and (high-tech) electronic tablet virtual manipulatives (Bouck et al., Ha & Fang), as well as comparing two-dimensional illustration against three-dimensional paper-based illustrations and objects (Bara et al.).

Beal and Rosenblum (2018) found that most students (88%) with visual impairments across grade levels demonstrated significantly higher scores on math test items when using the test administered on electronic tablet, in comparison with paper; only five percent of these students scored higher with paper-based tests. McLaughlin and Kamei-Hannan (2018) found that middle and high school students with visual impairments had mixed accuracy results between paper and electronic tablet formats, with about 10 percent difference or less in error incidence, concluding that such errors had no significant effect on comprehension. However, the study’s participants tended to read text moderately faster when using electronic tablets over when using paper. Further, across repeated sessions, students each tended to increase their reading speeds by five to 10 words per minute with the tablet, and decreased speed with paper. The researchers indicated that the participants’ increased reading speed was an important improvement in fluency. Although not comparing test scores, Kong and colleagues (2018) found that high school students without disabilities took significantly longer on average to complete science, reading, and math tests presented on tablets with touchscreen responding than on computer screens with keyboard responding. In specific analyses of item types, including traditional multiple-choice items and technologically-enhanced items such as fill-in-the-blank, they found little variation in the response time pattern, with small item effect sizes; the exception was that drag-and-drop items took essentially the same amount of time.

Bara and colleagues (2018) found that grade 1 students with visual impairments differed in their recall of narrative reading details, comparing the two-dimensional illustrations and the three-dimensional (tactile) images conditions: while all three students recalled more with illustrations than without, two children correctly recalled about 25 to 35 percent more details in the 3-D images condition, and the other child recalled slightly more details with the 2-D images. Of the types of story details, students recalled only object transformations better with 3-D images. The other related studies investigated the impact for middle school students of using manipulatives over not using them when responding to fraction-based math problems. Bouck and colleagues (2018) found that students with intellectual or learning disabilities performed better with manipulatives than without them, but that both the concrete or physical manipulatives and the virtual manipulatives presented on an electronic screen resulted in similar scores. Further, students took longer to complete the math tasks when using manipulatives than when not having them, yet they took similar amounts of time between the physical and virtual manipulatives. Ha and Fang (2018) found that study participants, who had no reported disabilities, scored over 20 percent higher on spatial skills with the manipulatives than without them. When comparing student performance by gender, male students scored about 23 percent higher on average than female students on the paper format (without the manipulatives tool); when both groups used the manipulatives tool, their scores significantly improved, yet the gender-based average score difference had decreased, with males scoring six percentage points higher than females.

We identified separate reportable findings on the impact of four unique accommodations—that is, each of these accommodations that were the focus of just one study. Effects of these four unique accommodations were examined by three studies:

One study (Cobb, 2018) described the impact of students with various disabilities using unspecified accommodations naturalistically (as assigned in their individualized education program/IEP plans) when completing state assessments. Results indicated that students seemed to improve in their English language arts scores but not their mathematics scores.

Findings for each of these unique accommodations are reported in Appendix F.

Perceptions about Accommodations

Seven studies (Beal & Rosenblum, 2018; Bone & Bouck, 2018; Bouck et al., 2018; Ha & Fang, 2018; Hansen et al., 2018; Kong et al., 2018; Pangatungan, 2018) provided findings on perceptions of accommodations (see Appendix F for details on these studies). Nearly all of them reported on students’ perceptions, although one study (Pangatungan) highlighted only the perceptions of students’ parents and educators. In addition to students’ perceptions, Beal and Rosenblum also detailed the perceptions of educators. Two studies (Beal & Rosenblum, Bone & Bouck) yielded students’ perceptions about specific accommodations, when compared to conditions of not using accommodations. The other four studies on student perceptions (Bouck et al., Ha & Fang, Hansen et al., Kong et al.) detailed both perceptions as well as preferences.

Students had positive perceptions of most of the accommodations. Students with visual impairments in grades 4–10 mostly (78%) expressed positive experiences with the tablet app, rather than reading math items on paper, while 15 percent preferred the paper-based format reportedly due to higher familiarity (Beal & Rosenblum, 2018). Similarly, students with learning disabilities, or both learning disabilities and attention difficulties, indicated liking the opportunity to use scientific calculators, and preferred using this accommodation over not having it available (Bone & Bouck, 2018). Students in grades 7–8 with learning disabilities or intellectual disabilities enjoyed both the concrete and virtual manipulatives over not using them (Bouck et al., 2018). Middle school students (without disabilities) expressed via survey questions using a 7–8 point scale mostly positive perceptions about the virtual and physical manipulatives. Students’ median responses were highest on ease of use and ease of learning to use, both rating a ‘6,’ or ‘agree.’ The median response on usefulness and satisfaction were both rated as ‘somewhat agree,’ or a ‘5’ (Ha & Fang, 2018). High school students and other youth with hearing impairments indicated their perceptions of the quality of the signing by the human: 60 percent of survey respondents endorsed either ‘very good’ or ‘excellent,’ with only one participant finding fault with the relatively fewer than desired pauses by the human when signing. In contrast, 10 percent of survey respondents rated the avatar’s simulated signing in one of the two highest ratings, with common criticisms including that the avatar’s facial expressions, mouthing of the words, and body movement were not sufficiently communicative (Hansen et al., 2018).

Students also indicated preferences between versions of some accommodations. Bouck and colleagues (2018) reported that two students preferred using the virtual manipulatives app while one had no preference. Ha and Fang (2018) found that most students (72%) preferred using both physical and virtual manipulatives, with students commenting that they thought the different manipulative types complemented each other, in that the physical touch connected them to the academic content, yet the computer-based manipulative quickly provided precise information about angles. Ha and Fang noted that most of the remaining students (19%) preferred using the virtual manipulative. Hansen and colleagues (2018) indicated that 29 of the 31 students in their study preferred the human presenting ASL. These researchers noted further that nearly all participants indicated their impressions that the English version of the math test would have been sufficient for them to answer the items correctly, and students suggested various ideas for improving the ASL supports for the math test. Unfortunately, Kong and colleagues (2018) presented very little student survey data from high school students without disabilities, limiting details about perceptions or preferences.

Other non-student participants indicated similarly positive perceptions in general, highlighting specific considerations. Teacher surveys indicated students had higher motivation when using tablets over paper, and interviews yielded that more teachers (78%) had the impression that students were more engaged with the test on the tablet, while 22 percent reported no difference in engagement with the test on the tablet than on paper (Beal & Rosenblum, 2018). Pangatungan (2018), a dissertation researcher, presented substantial detail from surveys and interviews of both parents and teachers of students with and without learning disabilities in grades 4–5. Deriving five themes and 12 sub-themes, the researcher found that oral delivery was supportive of students’ reading performance, including school grades and state tests, according to teachers. Teachers also perceived that their reading aloud to students can model and instruct decoding, while preventing students from becoming discouraged. Parents also indicated that their children need and benefit from having access to various content including academic content through either the parents reading to them or students using computer-based oral delivery of content when completing schoolwork at home.

Implementation and Use of Accommodations

Three studies (Beal & Rosenblum, 2018; Cobb, 2018; Kelly et al., 2018) had findings related to accommodations use and implementation issues. In two studies (Beal & Rosenblum, 2018; Kelly et al., 2018), researchers described patterns of accommodations use, while one study (Cobb, 2018) provided information about educators’ accommodations implementation practices.

Beal and Rosenblum (2018) found that teachers of students with visual impairments indicated that these students seemed to need little help overall, and only for very few math test items, when using the electronic tablet format. Students typically sought assistance with navigation on the tablet, explaining the math problems, computing the math, and discussing the graphics. There were no apparent differences in numbers of items or degree of need for these assistance requests between the braille-using students and those reading print. Kelly and colleagues (2018) reported, from group-wise comparisons, that both groups of students with visual impairments—school-aged children with amblyopia and children with non-amblyopic strabismus—took about 28 percent longer on average than their peers without visual impairments to complete the task of copying item responses from test booklets to answer forms. All three groups had similar rates of copying errors. The researchers expressed concern that students with visual impairments who were not blind, particularly those with substantially different vision in one eye over the other, can still have fine motor impairments. These students have demonstrable needs for, and benefits from, extended time or other accommodations, especially in high-stakes academic activities.

Cobb (2018) found that all educator participants, both teachers and administrators, responded similarly on a survey describing accommodations implementation issues. Mean rating responses to specific survey items, on a 3-point scale, ranged from 2.3 to 2.6, showing that the high school was implementing important elements of an inclusive education system, such as teachers having substantial knowledge of accommodations (with a rating of 2.3) and availing all students of accommodations during instruction (rating 2.6). Open-ended question responses were about 55 percent positive and 45 percent negative in terms of considering the needs of students with disabilities within the general education setting. Cobb concluded that, while accommodations, as a component of inclusive practices, were not flawlessly implemented in the first two years, the 2017 performance of the high school’s students with disabilities on the state ELA assessment increased to higher than the state average, after previously being below average, demonstrating a higher than (state) average rate of improvement. Cobb also noted that the students with disabilities at the high school had not improved in state math assessment performance during the same time frame, persisting in low mean performance that remained below the statewide math assessment score average.

Accommodations by Academic Content Assessments

As in previous reports, we analyzed research findings according to the academic content area included in each of the studies. The content areas, presented in terms of the number of studies including them were: mathematics (n=7), reading (n=5), science (n=1), and other language arts (n=1). For each content area, we examined the impact on assessment performance, perceptions about accommodations, and implementation and use patterns. This analysis included all 11 studies. One study (Kong et al., 2018) incorporated accommodations’ effects on three content areas: math, reading, and science assessments. Another study (Cobb, 2018) addressed implementation of accommodations in two areas, math and other language arts. (See Appendix F for a more detailed explanation of the findings of each study.)

Mathematics. All seven of the studies involving mathematics presented findings on the effects of accommodations on performance. Four studies (Beal & Rosenblum, 2018; Bone & Bouck, 2018; Bouck et al., 2018; Hansen et al., 2018) provided effects data only for students with disabilities, while two studies (Ha & Fang, 2018; Kong et al., 2018) did so only for students without disabilities. One study (Cobb, 2018) reported a secondary data analysis on extant data sets of math (and ELA) state assessment mean performance of students with disabilities. None of the math-related studies provided comparisons across groups of students with and without disabilities.

Two studies using math assessments examined the effects of manipulatives, both concrete or tactile tools as well as high-tech manipulatives presented on screens (Bouck et al., 2018; Ha & Fang, 2018), while two other studies presented the impact of electronic administration (Beal & Rosenblum, 2018; Kong et al., 2018). One study reported on the impact of scientific calculators (Bone & Bouck, 2018) and one study reported on the impact of ASL (Hansen et al., 2018).

Accommodations benefited the math performance of students with disabilities in three studies (Beal & Rosenblum, 2018; Bone & Bouck, 2018; Bouck et al., 2018), and students without disabilities benefited from using supports in one study (Ha & Fang, 2018). Beal and Rosenblum (2018) found that students with visual impairments in grades 4–10 scored significantly higher on pre-algebra story problems that were presented on electronic tablets than when they were presented on paper. Both electronic and paper formats provided either enlarged font or braille tools, based on how each participant typically read tests. Bone and Bouck (2018) found that all participants—grade 8 students with learning disabilities, attention problems, or both—scored higher on multi-step math computation items when using scientific calculators than when they did not. The participants also tended to complete more task steps when using calculators. Bouck and colleagues (2018) indicated that grade 7–8 students with learning or intellectual disabilities had higher scores on fraction-related math problems when using manipulatives than when not using manipulatives, yet the scores were essentially the same between the concrete manipulatives and the virtual manipulatives presented electronically. However, the students required more time to complete the math tasks when using manipulatives of either type than when not using them. Ha and Fang (2018) found that middle school students, who had no reported disabilities, scored higher on a norm-referenced spatial skill test (PSVT-R; cf. Yoon, 2011) associated with math reasoning when using a virtual and physical manipulatives tool than when completing the paper-based test without the tool. Further, the pattern of gender-linked baseline performance difference appeared to be nearly neutralized with the use of the manipulatives tool.

One study (Hansen et al., 2018) indicated that there was no significant difference between two versions of ASL; that is, high school students and other youth with hearing impairments who used ASL scored similarly on math items presented with the recorded human signing and the avatar-simulated signing. Finally, a study that did not measure score effects produced a negative finding on a different effect, between two versions of an accommodation: Kong and colleagues (2018) found that response times for high school students without disabilities on math (algebra) assessments were much higher, by 52 seconds for 10 items, on tablets with touchscreen responding than on computer screens with keyboard responding.

Six of the seven math-related studies—besides Cobb (2018)—yielded findings about perceptions and preferences regarding accommodations. All six studies (Beal & Rosenblum, 2018; Bone & Bouck, 2018; Bouck et al., 2018; Ha & Fang, 2018; Hansen et al., 2018; Kong et al., 2018) gathered data from students, and one of these studies (Beal & Rosenblum, 2018) also described teachers’ perceptions of accommodations. Students’ positive views were reported in five studies (Beal & Rosenblum, 2018; Bone & Bouck, 2018; Bouck et al., 2018; Ha & Fang, 2018; Hansen et al., 2018), and their preferences for one version of the accommodations over another were also noted in three studies (Bouck et al., 2018; Ha & Fang, 2018; Hansen et al., 2018).

Beal and Rosenblum (2018) found that students with visual impairments in grades 4–10 mostly (78%) expressed positive experiences with electronic administration, and 15 percent of them indicated that they would rather use the standard paper-based format. Beal and Rosenblum found that 78 percent of teachers had the impression that students were more engaged when taking the math test on the electronic tablet than on paper, and 22 percent perceived no difference in engagement between these formats. Bone and Bouck (2018) concluded that the students with disabilities in grade 8 increased in their positive views of using scientific calculators from the beginning to the end of the study. Bouck and colleagues (2018) found that all three students in grades 7–8 with learning or intellectual disabilities viewed manipulatives positively, with two preferring the virtual, screen-based manipulatives, and one having no preference. Ha and Fang (2018) concluded that middle school students with no reported disabilities gave relatively high median ratings for virtual and physical manipulatives on usability surveys: ‘6’ (on a 7-point scale) on ease of use and ease of learning, and ‘5’ on usefulness and satisfaction. Most students (72%) preferred using both types of manipulatives in combination, and 19 percent preferred using virtual manipulatives. Hansen and colleagues (2018) found that about 60 percent of high school students and other youth with hearing impairments thought the ASL presented by a recorded person was either “very good” or “excellent,” but that 10 percent rated ASL by an avatar in the same way. In fact, 29 of the 31 survey respondents preferred the human version, one preferred the avatar, and one had no preference. The avatar was reported not to adequately communicate ASL through facial expressions, mouthing of the words, and body movement; students’ additional suggestions were also detailed. Kong and colleagues (2018) gathered high school students’ survey data, yet presented very little detail on their perceptions or preferences. Cobb (2018) concluded, from results of an educator survey (cf., Stetson & Associates, Inc., 2014) in a specific high school, that the implementation of inclusive education policies and practices, including accommodations in instruction and assessment, did not seem to influence any increased performance on state math assessments for students with disabilities.

Reading. Of the four studies involving reading, three studies (Bara et al., 2018; Kong et al., 2018; McLaughlin & Kamei-Hannan, 2018) presented findings on the effects of accommodations on assessment performance. Two studies (Bara et al., McLaughlin & Kamei-Hannan) provided effects data only for students with disabilities, and one study (Kong et al.) presented findings on the effects on response times for students with no reported disabilities, as well as the students’ perceptions of responding to two electronic administration versions. The remaining study (Pangatungan, 2018) described the perceptions of teachers and parents of students with and without disabilities about oral delivery of reading content. None of the reading-related studies provided performance comparisons across groups of students with and without disabilities.

Accommodations benefited the reading assessment performance in both studies for students with disabilities; participants in both studies had visual impairments. Examining data from three grade 1 students with visual impairments, Bara and colleagues (2018) found that two children correctly recalled about 25 to 35 percent more details in the 3-D images condition than in either the 2-D images or no-illustration conditions. The other child recalled slightly more details with the 2-D images over the other conditions. Further, the children recalled the narrative story’s elements differently: more object transformation details were recalled when using 3-D illustrations, while character-related details and other information were recalled at similar rates across the test conditions. McLaughlin and Kamei-Hannan (2018) reported mixed accuracy results for middle and high school students with visual impairments when using enlarged font presented on standard paper test format and presented on electronic tablet format, with about 10 percent difference or less in error incidence. The researchers concluding that such errors had no significant effect on reading comprehension. However, the study’s participants tended to read text moderately faster when using electronic tablets in comparison to when using paper. In fact, after repeated tablet use, students each tended to increase their reading speeds by five to 10 words per minute. The researchers concluded that electronic administration facilitated participants’ increased reading speed, a distinct improvement in fluency.

Pangatungan (2018) indicated that teachers of grade 4–5 students with and without learning disabilities perceived that their reading aloud to students can model and instruct decoding, while preventing students from becoming discouraged. Parents of these same students indicated that their children need and benefit from having access to academic content through either the parents reading to them or students using computer-based oral delivery of content when completing schoolwork at home. The researcher concluded that oral delivery was supportive of students’ reading performance, including school grades and state tests, as well as reading homework.

Science. Only one study (Kong et al., 2018) provided findings for science. The finding simultaneously pertained to the comparison of two electronic administration response conditions during math and reading assessments, while not reporting details for the student perceptions of these accommodations. Kong and colleagues (2018) found that response times for high school students without disabilities on science assessments were much higher on tablets with touchscreen responding than on computer screens with keyboard responding. Further, this pattern was essentially the same across various test item types.

Other Language Arts. Only one study (Cobb, 2018) provided findings for English language arts (ELA). Cobb (2018) examined data from educators on the implementation of inclusive education policies and practices including accommodations in instruction and assessment at a specific high school, and completed a secondary data analysis on extant data sets of ELA (and math) state assessment mean performance of students with disabilities. The researcher concluded that, although accommodations were not flawlessly implemented in the first two years, the 2017 performance of the high school’s students with disabilities on the state ELA assessment increased to higher than the state average, after previously being below average, demonstrating a higher than (state) average rate of improvement.

Table of Contents

 

Discussion

This report is the second consecutive NCEO report that has provided a snapshot of accommodations research literature involving only K–12 students published in a single year. This report summarizes accommodations research published in 2018. This is a narrower focus than most previous NCEO accommodations research reports (Cormier et al., 2010; Johnstone et al., 2006; Thompson et al., 2002; Rogers et al., 2012; Rogers et al., 2014; Rogers et al., 2016; Rogers et al., 2019; Zenisky et al., 2007); those incorporated more than one year of research literature, and encompassed elementary, secondary, and postsecondary education settings. We highlighted the types of accommodations that were studied, the purposes of the research, the research type, data sources, characteristics of the independent and dependent variables under study, and comparability of findings between studies in similar domains, including by specific accommodations and their performance effects. We also examined findings by academic content area.

Mathematics and reading were the content areas most frequently addressed in the studies included in this analysis—together comprising nearly all of the studies that used content assessments. Science comprised more than one-tenth of the studies with content assessments, or almost one-tenth of all studies described in this report. These proportions are generally similar to those noted in previous NCEO accommodations research reports (Cormier et al., 2010; Johnstone et al., 2006; Thompson et al., 2002; Rogers et al., 2012; Rogers et al., 2014; Rogers et al., 2016; Rogers et al., 2019; Rogers et al., 2020; Zenisky et al., 2007). An exception is that math comprised a larger proportion of studies, while reading was addressed in a smaller proportion. Science continues to be a content area of interest in K–12 accommodations research. Students were the participant group in more than three quarters of the studies. Students with visual impairments including blindness were participants in about one-third of the studies reported, while students with learning disabilities participated in a smaller proportion of accommodations research in 2018 than has been typical in recent years. Other disability categories receiving attention by some studies included attention-related disabilities, deafness/hearing impairment, and intellectual disabilities; each of these categories comprised one study each.

Accommodations research has continued to generate substantial research activity and has expanded in breadth and depth. The number of studies we located increased across the span of NCEO’s accommodations research summary reports; for instance, in 2011–2012, there were 49 identified studies, in 2013–2014, there were 53 studies, and in 2015–2016, there were 58 studies. A substantial proportion of the studies examined in those previous reports highlighted research on the academic accommodations provided at institutions of higher education. Keeping in mind that the most recent NCEO report (Rogers et al., 2020) in the series addressed only one year, 2017, and was narrowed to the context of the U.S. K–12 school system, the 14 studies described were consistent with a continually increasing trend for this topic in research literature. Further, the current report also addressed one year of accommodations research (published in 2018) within the U.S. K–12 context; it includes 11 studies, a number generally similar to the previous one-year U.S. K–12 accommodations research.

Researchers have continued to explore a wide range of topics related to assessment accommodations, including the comparison of the effects of differing versions of accommodations, such as providing physical or virtual manipulatives, and human-signed or simulated ASL. Studies also continued to address multiple purposes and to examine various types of data. Along with investigating effects on content tests, researchers inquired about perceptions of students with disabilities and educators about accommodations, and students’ use patterns and educators’ implementation practices surrounding accommodations.

Both quantitative analyses and thematic exploration of qualitative data have yielded findings that can inform and improve considerations for addressing students’ needs. For instance, Beal and Rosenblum (2018) investigated potential differences between traditional paper-based tests and electronically-administered tests for students with visual impairments, and discussed issues related to motivation and persistence during test-taking. Similarly, Pangatungan (2018) inquired about the perceptions of the parents and teachers of elementary students with learning disabilities, seeking to gain deeper understanding from important others about student engagement and motivation. These issue-driven inquiries can offer additional insights and perspectives about the provision of assessment accommodations to students with disabilities who need them.

Interest in research on accommodations continues. This research can inform states as they consider their accommodations policies. It can also provide information for others interested in assessment accommodations.

Table of Contents

 

References

Report References

(References in the report to papers that were part of the 2018 accommodations research analysis are not included in this list. They are in the separate list titled 2018 K–12 Accommodation References.)

Birch, E., & The Retina Foundation of the Southwest. (2018). Randot preschool stereoacuity test. Stereo Optical Company, Inc.

Cormier, D. C., Altman, J. R., Shyyan, V., & Thurlow, M. L. (2010). A summary of the research on the effects of test accommodations: 2007–2008 (Technical Report 56). University of Minnesota, National Center on Educational Outcomes.

Connolly, A. J. (2007). KeyMath-3 diagnostic assessment: Manual forms A and B. Pearson.

Johns, J. L. (2012). Basic reading inventory (11th ed.). Kendall Hunt Publishing.

Johnstone, C. J., Altman, J., Thurlow, M. L., & Thompson, S. J. (2006). A summary of research on the effects of test accommodations: 2002 through 2004 (Technical Report 45). University of Minnesota, National Center on Educational Outcomes.

Kaufman, A. S., & Kaufman, N. L. (2004). Kaufman test of educational achievement (2nd ed.) (KTEA™-II). Pearson.

Lund, A. M. (2001). Measuring usability with the USE Questionnaire. STC Usability SIG Newsletter, 8(2), 3–6.

Rogers, C. M., Christian, E. M., & Thurlow, M. L. (2012). A summary of the research on the effects of test accommodations: 2009–2010 (Technical Report 65). University of Minnesota, National Center on Educational Outcomes.

Rogers, C. M., Lazarus, S. S., & Thurlow, M. L. (2014). A summary of the research on the effects of test accommodations, 2011–2012 (Synthesis Report 94). University of Minnesota, National Center on Educational Outcomes.

Rogers, C. M., Lazarus, S. S., & Thurlow, M. L. (2016). A summary of the research on the effects of test accommodations: 2013–2014 (NCEO Report 402). University of Minnesota, National Center on Educational Outcomes.

Rogers, C. M., Thurlow, M. L., Lazarus, S. S., & Liu, K. K. (2019). A summary of the research on effects of test accommodations: 2015–2016 (NCEO Report 412). University of Minnesota, National Center on Educational Outcomes.

Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland adaptive behavior scales (2nd ed.). Pearson.

Stereo Optical Company, Inc. (2018). Random dot stereo butterfly. Author.

Stetson & Associates, Inc. (2014). Assessment of school practices related to inclusive education. Author.

Thompson, S., Blount, A., & Thurlow, M. (2002). A summary of research on the effects of test accommodations: 1999 through 2001 (Technical Report 34). University of Minnesota, National Center on Educational Outcomes

Wechsler, D. (2004). Wechsler intelligence scale for children. Pearson Assessments.

Yoon, S. Y. (2011). Revised Purdue Spatial Visualization Test: Visualization of rotations (Revised PSVT:R) [Psychometric instrument: Purdue University].

2018 K–12 Accommodation References

Bara, F., Gentaz, E., & Valente, D. (2018). The effect of tactile illustrations on comprehension of storybooks by three children with visual impairments: An exploratory study. Journal of Visual Impairment & Blindness, 112(6), 759–765. https://doi.org/10.1177/0145482X1811200610

Beal, C. R., & Rosenblum, L. P. (2018). Evaluation of the effectiveness of a tablet computer application (app) in helping students with visual impairments solve mathematics problems. Journal of Visual Impairment & Blindness, 112(1), 5–19. https://doi.org/10.1177/0145482X1811200102

Bone, E. K., & Bouck, E. C. (2018). Evaluating calculators as accommodations for secondary students with disabilities. Learning Disabilities: A Multidisciplinary Journal, 23(1), 35–49. https://doi.org/10.18666/LDMJ-2018-V23-I1-8437

Bouck, E., Shurr, J., Bassette, L., Park, J., & Whorley, A. (2018). Adding it up: Comparing concrete and app-based manipulatives to support students with disabilities with adding fractions. Journal of Special Education Technology, 33(3), 19–206. https://doi.org/10.1177/0162643418759341

Cobb, P. (2018). Examining the efficacy of inclusive practices and its impact on the academic achievement of high school students with mild to moderate disabilities. Dissertation Abstracts International: Section A. Humanities and Social Sciences, 79/11(E). https://pqdtopen.proquest.com/doc/2071341048.html?FMT=ABS

Ha, O., & Fang, N. (2018). Interactive virtual and physical manipulatives for improving students’ spatial skills. Journal of Educational Computing Research, 55(8), 1088–1110. https://doi.org/10.1177/0735633117697730

Hansen, E. G., Loew, R. C., Laitusis, C. C., Kushalnagar, P., Pagliaro, C. M., & Kurz, C. (2018). Usability of American Sign Language videos for presenting mathematics assessment content. Journal of Deaf Studies and Deaf Education, 23(3), 284–294. https://doi.org/10.1093/deafed/eny008

Kelly, K. R., Jost, R. M., De La Cruz, A., & Birch, E. E. (2018). Multiple-choice answer form completion time in children with amblyopia and strabismus. JAMA Ophthalmology, 136(8), 938–941. https://doi.org/10.1001/jamaophthalmol.2018.2295

Kong, X., Davis, L. L., McBride, Y., & Morrison, K. (2018). Response time differences between computers and tablets. Applied Measurement in Education, 31(1), 17–29. https://doi.org/10.1080/08957347.2017.1391261

McLaughlin, R., & Kamei-Hannan, C. (2018). Paper or digital text: Which reading medium is best for students with visual impairments? Journal of Visual Impairment & Blindness, 112(4), 337–351. https://doi.org/10.1177/0145482X1811200401

Pangatungan, M. (2018). The impact of read-aloud accommodations of fourth- and fifth-grade elementary students with and without learning impairment: A descriptive case study. Dissertation Abstracts International: Section A. Humanities and Social Sciences, 80(05)E. http://search.proquest.com/docview/2169127741/abstract

Table of Contents

 

Appendix A

Research Characteristics for K–12 Studies in 2018

Authors Publication
Type
Research
Type
Research Design Data
Collection
Source
Collection
Instrument
Bara et al. Journal Quantitative Descriptive Quantitative Primary Observations
Beal & Rosenblum Journal Mixed Quasi-experimental Primary Interview, Survey, Test
Bone & Bouck Journal Quantitative Experimental Primary Observations, Survey, Test
Bouck et al. Journal Mixed Experimental Primary Interview, Observations, Test
Cobb Dissertation Quantitative Descriptive Quantitative Primary1 Survey, Test
Ha & Fang Journal Mixed Quasi-experimental Primary Survey, Test
Hansen et al. Journal Mixed Quasi-experimental Primary Interview, Test
Kelly et al. Journal Quantitative Descriptive Quantitative Primary Observations, Test
Kong et al. Journal Mixed Quasi-experimental Primary Observations, Survey, Test
McLaughlin & Kamei-Hannan Journal Quantitative Quasi-experimental Primary Observations, Test
Pangatungan Dissertation Qualitative Descriptive Qualitative Primary Interview, Observations, Survey

1 The researcher gathered primary data, yet also employed secondary (extant) state assessment data from the high school and the statewide assessment scores for analyses.

Table of Contents

 

Appendix B

Research Purposes for K–12 Studies in 2018

Authors Stated Research Purpose E
f
f
e
c
t
s
(SwD)
E
f
f
e
c
t
s
(non)
E
f
f
e
c
t
s
(both)
P
e
r
c
e
p
t
i
o
n
s
I
m
p
l
e
m
e
n
t/
U
s
e
I
s
s
u
e
s
R
e
v
i
e
w
I
t
e
m
s
T
e
s
t
V
a
l
i
d
i
t
y
T
O
T
A
L
(P+Xs)
Bara et al. Investigate the effects of tactile illustrations during oral delivery of narrative content on reading comprehension for grade 1 students with visual impairments, with students providing oral responses on details.
       
      1
Beal & Rosenblum Investigate the effects of an electronic tablet app-based format compared to paper-based large print or braille for elementary, middle, and high school students with visual impairments when completing test-like math applied tasks; inquire about the perceptions of students on their experiences and preferences when performing pre-algebra content tasks, and about teachers’ perspectives of students’ relative independence; discuss issues related to motivational factors and to usability details for the improvement of the app. P

X

    4
Bone & Bouck Investigate the effects of scientific calculators, in combination with extended time and multiplication charts for grade 8 students with math-related disabilities on math test-like tasks involving computation and applied math problems; inquire about the perceptions of students on using calculators for math problem solving. P                 2
Bouck et al. Investigate the effects of virtual electronic tablet-based manipulatives compared to physical manipulatives and no manipulatives for grade 7-8 students with intellectual or learning disabilities when completing test-like math tasks involving fractions; inquire about the perceptions of students on using different types of manipulatives during math task completion. P  
            2
Cobb Describe educators’ practices and evaluate their relative use, of accommodations, along with other inclusive practices in a high school; investigate the effects of accommodations and other inclusive practices on the performance of students with disabilities in a California high school on state assessments through the comparison of an extant data set of the students’ English language arts and mathematics assessment scores against the statewide assessment performance of students with disabilities; summarize related research literature. X


P
X


3
Ha & Fang Investigate the effects on math-related spatial skills tasks of using a virtual and physical manipulatives (VPM) technology tool, in comparison with performance on a pre-test using two-dimensional renderings of geometric shapes, for grade 8 students without disabilities; inquire about the perceptions of students without disabilities (through a usability survey and additional questions) on their experiences and preferences when using the virtual and physical manipulatives tool, when using physical manipulatives, and when using virtual manipulatives.
P
X





2
Hansen et al. Investigate the effects on student performance on a secondary basic algebra assessment items presented in American Sign Language (ASL) by a recorded human signer and an avatar signer, in comparison to performance on an English written version of the assessment for students with hearing impairments or deafness who used both ASL and English; inquire about the perceptions of Deaf or hard-of-hearing students on their experiences and preferences when using the differently-presented ASL versions, including the clarity of the signing. P

X





2
Kelly et al. Report on use patterns, specifically completion times, of students with visual impairments and students without visual impairments, when performing test-related tasks using a standard multiple-choice answer sheet, which can inform potential needs for individual administration and timed administration by students with visual impairments.



P




1
Kong et al. Investigate the effects of two different electronically-administered test versions on multiple academic content areas, requiring either keyboard responding on a computer or touchscreen responding on a tablet, on item response times for high school students without disabilities; inquire about the perceptions of participants on their experiences and preferences in item responding on different electronic test administration platforms.
P
X





2
McLaughlin & Kamei-Hannan Investigate the effects of low-tech/paper-based print enlargement compared to electronic tablet-based magnification and other technological visual supports on reading speed and production errors for students with low vision who are print readers; discuss issues regarding silent and oral reading performance tasks and reading skill development. P



X



2
Pangatungan Inquire about the perceptions of parents and teachers of elementary students with and without learning disabilities on the “read-aloud accommodation” (oral delivery live and in-person), including impacts on academic performance and classroom behavior; summarize related research literature; discuss issues including the potential impact on students’ engagement and motivation, and the variety of views across the purposively selected group of participants, such as between teachers and parents, and between parents with children with and without learning disabilities.


P
X X


3

KEY

Effects [SwD] Examine effects of accommodations on assessment scores
[only students with disabilities]
Effects [non] Examine effects of accommodations on assessment scores
[only students without disabilities]
Effects [both] Examine effects of accommodations on assessment scores
[both students with and without disabilities]
Perceptions Study/compare perceptions and preferences about use
Implement/Use Report on implementation practices and accommodations use
Issues Discuss issues related to accommodations
Review Review or summarize research on test accommodations
Items Compare test items across assessment formats
Test Evaluate test structure
Validity Investigate test validity under accommodated conditions
TOTAL (P+Xs) Total of all purposes, including primary and other purposes
P Primary Purpose
X Other Purpose

Table of Contents

 

Appendix C: Instrument Characteristics for K–12 Studies in 2018

Table C-1. Specific Instrument Sources

Authors Instrument Sources and Descriptions Number of Sources
Bara et al. Author (observation): Numbers of verbal student-initiated interactions with the instructor, numbers of correct details recalled, and percentages of matches between text elements and illustrations, as measured by the researcher team from video-recordings. 1
Beal & Rosenblum Author (interview, survey): Teacher survey rating the degrees of support that their students sought while completing math items, and the degrees of students’ motivation. Phone interview protocol with students and their teachers to report on their experiences and preferences, and observations, regarding the accommodations. Researcher (test): Sets of 24 mathematics story problems (also called word problems) aligned to Common Core State Standards for grade 6 math, incorporating pre-algebra content, were completed by student participants under the two task completion conditions. 2
Bone & Bouck Author (survey, observation): 9-item student survey that explored the social validity of using calculators both at the beginning and end of the study. Participants’ use or non-use of calculators when provided was observed and documented by researchers. Researcher (test): Mathematics problems based on Common Core State Standards were presented in five testing sessions; math computation items comprised the tests during baseline and intervention phases, then math word problems were given to check for skill generalization. Numbers of correct answers and numbers of items attempted were documented; partial credit was given for partial completion of items (such as evidence of steps of task completed). 2
Bouck et al. Author (observation, interview): Assessment task completion time was measured with a stopwatch app, observed and documented by researchers. Post-assessment interview questions pertaining to social validity, including students’ preferred manipulative type. Researcher (test): Sets of math tests with five items on the addition of fractions with unlike denominators comprised the accuracy measure; performance score was the number of correct answers out of five questions. Task independence was measured based on researchers’ observations of the proportion of total task steps completed without seeking or requiring prompting by test administrators, as documented on system of least prompts forms. Norm-ref Ach: (for screening purposes) Kaufman Test of Educational Achievement, Second Edition (KTEA™-II; Kaufman & Kaufman, 2004). Norm-ref Ability: (for screening purposes) Relevant subtests of Key Math-3 Diagnostic Assessment (KeyMath-3; Connolly, 2007). Wechsler Intelligence Scale for Children, Fourth Edition (WISC, 4th ed.; Wechsler, 2004). Other: (for screening purposes) Vineland Adaptive Behavior Scales, Second Edition (Vineland-II; Sparrow, Cicchetti, & Balla, 2005). 5
Cobb Researcher (survey): A questionnaire (Stetson & Associates, Inc., 2014) adapted by the researcher into a 31-item rating survey was completed by teachers and school administrators for rating the degree to which accommodations, part of a total of 11 inclusionary practices, were implemented in a California high school. Educators also reported their demographics and responded to a subset of open-ended questions in the survey. State Test: The extant data set of English language arts and mathematics assessments, the 2017 Smarter Balanced Assessment Consortium (SBAC) California assessment, from the high school was compared to the statewide assessment results. 2
Ha & Fang Author (survey): Student survey on preferences among the manipulative conditions and reasons for the ratings. Researcher (survey): Student survey of perspectives on Usefulness, Satisfaction, and Ease of Use (USE; Lund, 2001). Norm-ref Ability: Student participants’ scores on the Purdue Spatial Visualization Test: Revised (PSVT-R; Yoon, 2011), a 30-item multiple-choice assessment, were analyzed. The researchers noted that the spatial skill of mental rotation is associated with STEM (science, technology, and math) reasoning. 3
Hansen et al. Author (interview): A series of interview questions on student participants’ experience completing the test items using different ASL signers (human vs. avatar), including their perceptions about clarity of the signing, as well as their preferences for either signer version, and their suggestions for improving usability. Researcher (test): Mathematics test items were drawn from a set of practice items for preparing for a standardized math test, and were linked to standards expected of a student at the middle school or early high school level. Three math test items included graphics. 2
Kelly et al. Author (observation): Participants’ completion times were observed and documented by researchers using a stopwatch. State Test: The accuracy of students’ test-taking behavior—transferring 40 already answered correct responses of the 2013 Texas Assessment of Academic Readiness grade 3 reading achievement test (publicly available version) from the test booklet to a blank Scantron “bubble sheet”—was measured based on the researchers’ comparisons of the participants’ transcribed answer forms to the actual provided item answers. Other: (for screening purposes) Participants’ visual impairments were prescreened for stereoacuity with the Randot Preschool Stereoacuity Test (Birch & The Retina Foundation of the Southwest, 2018) and Random Dot Stereo Butterfly test (Stereo Optical Company, Inc., 2018). 3
Kong et al. Author (observation, survey): Student participants’ item response times were observed and documented by researchers. 10-question student survey on their familiarity with devices and on their perceptions of test-taking during the study. Researcher (test): Test items for (high school level) English II, Biology, and Algebra I were presented as traditional multiple-choice items and technologically-enhanced items: multiple select, drag and drop, fill in the blank, graph point, hot spot, and inline choice (from drop-down box). [Participants’ correctness scores were calculated for solution behavior and rapid-guessing behavior, not as primary data for comparison, but rather, for the purpose of checking for validity of these different test-taking behaviors.] 2
McLaughlin & Kamei-Hannan Author (observation): Researchers documented observations, tracking completion time associated with reading speeds (in words per minute), as well as accuracy, with number of errors in pronunciation. Norm-ref Ach: The Johns Basic Reading Inventory (Johns, 2012) was used to independently check the reading skill levels for the participants. 2
Pangatungan Author (interview, survey, observation): Perceptions of parents and teachers were captured through a semi-structured interview protocol, documented in the researcher’s notes, with full quotes validated by interviewees. A survey rating the relative impact of oral delivery on students’ overall reading performance was also completed by the parents and teachers of students with and without learning disabilities and other difficulties. Field notes comprising the researcher’s observations served to authenticate the other data sources. 1

KEY:

Instrument Sources Type Abbreviations Number of Studies
Non-Academic Protocols or Surveys Developed by
Study Author/s
Author (Interview/
Observation/Survey)
10
Surveys or Academic Tests Developed by
Professionals or Researchers through Work Outside
of Current Study
Researcher Test 7
State Criterion-referenced Assessment State Test 2
Norm-referenced Academic Achievement Measures Norm-ref Ach 2
Norm-referenced Cognitive Ability Measures Norm-ref Ability 2
Other Other 2

Table C-2. Content Areas Assessed

Authors Math Reading Writing Other
LA
Science Social
Studies
TOTAL
Bara et al.




1
Beal & Rosenblum




1
Bone & Bouck




1
Bouck et al.




1
Cobb

a

2
Ha & Fang




1
Hansen et al.




1
Kelly et al.




1
Kong et al.


3
McLaughlin & Kamei-Hannan




1
Pangatungan




1
TOTAL 7 5 0 1 1 0

Note: This table encompasses the studies (N=11) which used assessments or tests on academic content area/s or cognitive skills.

a In this study, other LA = identified by Smarter Balanced Assessment as English language arts, with both reading comprehension of literary and informational texts, and writing—producing effective and well-grounded writing.

Table of Contents

 

Appendix D

Participant and Sample Characteristics for K–12 Studies in 2018

Authors Unit of
Analysis
Sample Size Percent of
Sample with
Disabilities
Grade /
Education
Level
Disability
Categories
Included in
Sample
Bara et al. Students 3 100% grade 1 VI
Beal & Rosenblum Students;
Educators
73 59% grades 4-10 (n=43);
teachers (n=30)
VI, N/A
Bone & Bouck Students 5 100% grade 8 LD; LD & AP
Bouck et al. Students 3 100% grades 7-8 ID, LD
Cobb Educators 36 0% educators
(teachers and
others, at high
school)
N/A
Ha & Fang Students 44 0% middle school None
Hansen et al. Students 31 100% high school (n=12);
postsecondary (n=19)
HI
Kelly et al. Students 85 76% 8-6-12.1 years
(grades 3-7)
VI, None
Kong et al. Students 964 0% high school None
McLaughlin & Kamei-Hannan Students 3 100% middle to high
school
VI
Pangatungan Educators;
Parents
21 0% teachers and
parents (of
students in
grades 4-5)
N/A

KEY

AP Attention problem
HI Hearing impairment/deafness
ID Intellectual disability
LD Learning disabilities
VI Visual impairment/blindness
N/A Non-students
None No disabilities

Table of Contents

 

Appendix E

Accommodations Studied for K–12 Studies in 2018

Table E-1. All Accommodations by Study

Author/s Accommodation/s (CATEGORY)
Bara et al. Tactile graphics/illustrations (PRESENTATION)
Beal & Rosenblum Electronic administration (PRESENTATION, EQUIPMENT, RESPONSE)
Bone & Bouck Calculator (RESPONSE)
Bouck et al. Technological aid (concrete/tactile and virtual manipulatives) (EQUIPMENT)
Cobb Not specified
Ha & Fang Technological aid (physical/tactile and virtual manipulatives) (EQUIPMENT)
Hansen et al. Signed administration (using American Sign Language) (PRESENTATION)
Kelly et al. Format; Individual setting (PRESENTATION, SETTING)
Kong et al. Electronic administration (PRESENTATION, EQUIPMENT, RESPONSE)
McLaughlin &
Kamei-Hannan
Electronic administration, Large print/magnification, Font/background contrast
(PRESENTATION, EQUIPMENT, RESPONSE)
Pangatungan Oral delivery, live/in-person; Oral delivery, text-to-speech (PRESENTATION)

Table E-2. Presentation Accommodations Itemized by Study

Author/s Electronic
adminis-
tration
Format Large print/
magnification
Oral delivery,
live/in-person
Signed
adminis-
tration
Tactile
graphics/
illustrations
Text-to-speech
device/software
TOTAL
Bara et al.  


    1
Beal & Rosenblum  
        1
Hansen et al.    
      1
Kelly et al.  
        1
Kong et al.  
        1
McLaughlin & Kamei-Hannan           2
Pangatungan
 
    2
TOTAL studies (of 7) 3 1 1 1 1 1 1

Table E-3. Equipment Accommodations Itemized by Study

Author/s Electronic
administration
Technological
aid
TOTAL
Beal & Rosenblum
1
Bouck et al.   1
Ha & Fang   1
Kong et al.   1
McLaughlin & Kamei-Hannan   1
TOTAL studies (of 5) 3 2

Table E-4. Response Accommodations Itemized by Study

Author/s Calculator Electronic
administration
TOTAL
Beal & Rosenblum   1
Bone & Bouck   1
Kong et al.   1
McLaughlin & Kamei-Hannan   1
TOTAL studies (of 4) 1 3

Table E-5. Setting Accommodations Itemized by Study

Author/s Individual TOTAL
Kelly et al. 1
TOTAL studies (of 1) 1

Table of Contents

 

Appendix F

Findings for K–12 Studies in 2018

Authors Findings Statement E
f
f
e
c
t
s
P
e
r
c
e
p
t
i
o
n
s
I
m
p
l
e
m
e
n
t/
U
s
e
V
a
l
i
d
i
t
y
C
o
n
t
e
n
t
Bara et al. Grade 1 students with visual impairments verbally engaged differently when the stories were read to them without illustrations compared to the two illustration conditions. Few or no interactions occurred when the story had no illustrations. In contrast, all three children made comments to the story reader several times when provided either two-dimensional (2-D) or three-dimensional (3-D) illustrations. With the 2-D images, two children commented or asked questions to verify words or understand the illustrations. They spoke less with the 3-D images, yet when they did, they indicated understanding the illustrations rather than asking questions. One child had a much larger proportion of matches between text and tactile exploration of the relevant 3-D illustrations in comparison to the 2-D images. The other two children had a somewhat smaller percentage of such matches with the 3-D images than with the 2-D images. Two children correctly recalled about 25–35% more details in the 3-D images condition than in the 2-D images or no-images conditions, while the other child recalled slightly more details with the 2-D images over the other conditions. More object transformation details were recalled when using 3-D illustrations, while character-related details and other information were recalled at similar rates across the test conditions. X


R
Beal & Rosenblum Performance comparisons for elementary, middle, and high school students with visual impairments who used either large print or braille presented on paper versus on an electronic tablet showed differences. Most (88%) scored higher on the electronic tablet, 5% scored higher on paper, and 7% scored similarly on both. These performance differences occurred throughout the students’ grade levels. Students were observed using various features on the tablet app in different frequencies. Glossary was used by 19% of braille users and 11% of print users. Teachers rated the degree of assistance they provided at students’ request. Assistance included navigation on the tablet, explaining the math problems, computing the math, and discussing the graphics. Teachers mostly indicated being asked for assistance for very few test items, and when asked for help, students most often sought a little help (1 on a scale of 0 to 4). Braille users and print users sought help with similar numbers of items and to similar degrees. Teacher surveys indicated ratings of higher student motivation when using tablets over paper. During interviews, more teachers (78%) shared the impression that students were more engaged with the test on the tablet, and 22% reported no engagement differences. Most students (78%) indicated their preference to use the tablet app, and 15% preferred the familiarity of their usual paper-based format. The remainder had no strong preference. X X X
M
Bone & Bouck Grade 8 students with learning and attentional disabilities mostly attempted most problems whether using scientific calculators or not. Students all scored higher with calculators; individual participant score patterns were reported. Students tended to complete more steps of problems, with individual variations for 2-step, 3-step, and 4-step problems. Students chose to use or not use the calculator at differing rates for steps of each of the five items when it was available, with a range of using calculators between about 40% and 80% of the time. The survey results indicated that the social validity of calculator use was higher overall at the study’s end. X X

M
Bouck et al. The accuracy of responses of middle school students with intellectual or learning disabilities using the concrete and app-based (virtual) manipulatives was equal, and these scores were higher than when using no manipulatives. Participants took longer to complete tasks when using manipulatives in general, yet concrete and virtual manipulatives took similar task time. The differences in independence were slight, but differed by student: two were more independent with the concrete manipulatives and one with the virtual manipulatives. Interview results showed that the participants enjoyed both the concrete and virtual manipulatives, but two preferred using the app while one had no preference. Anecdotal observations during math task completion were consistent with participants’ interview data. X X

M
Cobb All educator survey ratings (on a 3-point scale) averaged 2.6 across all inclusive practices, and permitting instructional accommodations averaged 2.6, while actively availing all students of accommodations was rated 2.3, and teachers’ accommodations knowledge was rated 2.3. Open-ended question responses were about 55% positive and 45% negative for considering the needs of students with disabilities within the general education setting. Open-ended responses on inclusive practices such as accommodations suggested that 19% believed that they were being implemented, 42% believed that the practices in place needed improvement, and 38% thought that they were not in place. Teachers and administrators has similar responses on instructional setting and practices. ABC High School’s ELA assessment performance by students with disabilities in 2017 was higher than the statewide average, after the ELA scores were lower than the state average in 2015. The increase demonstrated a higher than (state) average rate of improvement. However, the relatively low math performance of ABC High School’s students with disabilities in 2017 was essentially the same as the state-wide average for 2015—indicating no apparent change with the implementation of accommodations and other inclusion-related efforts. X
X
M, ELA
Ha & Fang The independent samples comparison showed a significant change in mean spatial skills performance score of nearly 4 points, a learning gain of about 21% for students using the virtual and physical manipulatives (VPM) tool. The paired sample comparison indicated a significant and even larger gain in spatial skills performance. By gender group, male students scored significantly higher in spatial skills than female students on the pre-test in paper format, by about 23%. When using the VPM tool, these score differences by gender decreased to 6%. VPM tool use benefited both males and females, yet supported greater improvements in spatial skill performance for female students. These middle school students (no disabilities reported) expressed mostly positive perceptions about the manipulatives. Students’ median response on usefulness was a ‘5’ on a 7-point scale, indicating “somewhat agree.” Students’ median response on ease of use was ‘6,’ indicating “agree,” and ease of learning how to use was also ‘6.’ Students’ median response on satisfaction was ‘5,’ yet varied by survey item. For instance, most students (n=25) indicated that manipulatives were fun to use, yet few (13%) indicated needing the manipulatives as a support. Preferences—for using either physical manipulatives, virtual manipulatives, or both types—indicated a strong preference for using both, by 72% of respondents. Most of the remaining students (19%) preferred using the virtual manipulative. Many students commented thinking that the different manipulative types complemented each other: physical touch connected them to the academic content, yet the computer-based manipulative quickly provided precise information about angles. X X

M
Hansen et al. The scores of students with hearing impairments were not significantly different between the two ways that ASL was presented, by the video recording of a human signer versus the simulated signing by an avatar. When subsequently presented with the English version of the math questions, participants’ average scores were significantly higher than the two ASL versions, with more participants scoring correctly on seven of the nine full English items than the signed items. The math performance scores and the accuracy of their translations of the ASL into English were highly correlated with one another, but the signed version and translation scores were not significantly correlated with one another. The implication is that ASL by human signer or avatar were not significantly different from one another in being comprehended by students. On a five-point scale, participants rated the signing higher in quality for the human signer over the avatar signer, with about 60% indicating the human signing was ‘very good’ or ‘excellent,’ and about 10% indicating those same ratings for the avatar signing. Of the 31 participants, 29 preferred the human’s ASL, one preferred the avatar, and one had no preference. Participants noted that the avatar’s communication was limited, due to the avatar’s facial expressions, mouthing of the words, and body movement. One participant noted that the human signer did not pause enough. Nearly all participants indicated their impressions that the English version of the math test (without ASL) would have been sufficient for them to answer the items correctly. Participants suggested various improvements for the ASL versions of the math test. X X

M
Kelly et al. The task of documenting multiple choice test answers on a bubble sheet answer form for school-age children with visual impairments required more time than for their peers without visual impairments. In groupwise comparisons for children with two different visual impairments and children without visual impairments, children with amblyopia and children with non-amblyopic strabismus took about 28% longer on average to complete the task of copying item responses from test booklets to answer forms. The groups of children with these two visual impairments took essentially the same amount of time on average to complete the task. There were no significant differences in rates of making copying errors across the three groups. The researchers remarked that students with substantially different vision in one eye over the other can still demonstrate fine motor impairments, yet accommodations such as extended test time have not been permitted because of having sufficient vision when using both eyes. Concern was noted for recognizing the needs of students with visual impairments but without blindness regarding access to accessibility and accommodations in high-stakes academic activities.

X
R
Kong et al. Relatively small but still significant response time differences were found between test devices: high school students (without disabilities) took longer to complete test items presented on tablets with touchscreen responding than on computer screens with keyboard responding. The tablet-based test section in science took about 100 seconds longer, in comparison to the computer-based test; the differences in reading tablet items took 58 seconds longer, and math tab-let items took 52 seconds longer. Specifically, tablet-based items across various item types took statistically significantly longer to complete on average, except that drag-and-drop items took no longer whether completed on touchscreens or on keyboards. The item type response time differences between tablet and computer were less than 10 seconds per item, with small effect sizes. Ethnicity and gender subgroups had similar response time differences, taking longer with tablet-based items in the different academic content areas. All of these patterns were shown after removing the response time data for test items that fit “rapid-guessing” patterns. Rapid guessing behavior was estimated at 0% to 14% for the tablet format and 0% to 19% for the computer format. Very little student survey data were provided in terms of findings. X X

M,R,S
McLaughlin & Kamei-Hannan In terms of reading speed, a component of fluency, all middle to high school students with visual impairments tended to read text moderately faster when using the iPad compared with paper, for both silent and oral reading. Across sessions, students each increased their reading speeds by five to 10 words per minute with the iPad format, and decreased speeds with the paper format. Individual participants’ data were complex. Only one of the three participants chose to change the font size (from 24-point to 27-point) and color contrast, indicating the changes were more efficient and gave him less fatigue. This student, Cody, completed 50% more reading tasks than the others, yet at a slightly slower average reading speed than the others. In terms of accuracy, based on the number and type of errors, students had mixed results between paper and electronic tablet formats. The rate of errors was not significantly different by format, either overall (about 10% or less) or in error type. The researchers concluded that errors had no significant effect on comprehension. Reading comprehension had a maximum score of 40, and participants’ scores were all at or very close to this maximum, and did not vary by reading task or condition. Tablet-based reading had essentially no impact on reading comprehension performance. The researchers noted, however, that the participants’ increased reading speed was an important improvement in fluency. X


R
Pangatungan The dissertation researcher sought to deeply understand the perspectives of parents and teachers of students with and without learning disabilities in grades 4–5. After integrating data from surveys and interviews on participants’ impressions of student performance on various academic activities including classroom and state assessments, the researcher reported five themes and 12 sub-themes, including some from teachers’ perceptions and different ones from parents’ perceptions. Teachers indicated that oral delivery was supportive of students’ reading performance, including school grades and state tests. Teachers noted that oral delivery allows students the opportunity to comprehend academic content rather than struggling with decoding alone. Teachers indicated that reading aloud to students can model and instruct decoding, while preventing students from becoming discouraged. Parents indicated that their children need and benefit from access to various content including academic content through either the parents reading to them or having computer-based oral delivery of content when completing schoolwork at home. Parents indicated that they value the access that their children with learning disabilities have to the oral delivery accommodation during state assessments.
X

R
TOTAL   9 7 3 0 11