Technical Report 66

Test Administrators' Perspectives on the Use of the Read Aloud Accommodation in Math on State Tests for Accountability

Jennifer R. Hodgson, Sheryl S. Lazarus, Lynn M. Price, Jason R. Altman, Martha L. Thurlow

October 2012

All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Hodgson, J. R., Lazarus, S. S., Price, L. M., Altman, J. R., & Thurlow, M. L. (2012). Test administrators' perspectives on the use of the read aloud accommodation in math on state tests for accountability (Technical Report 66). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Acknowledgments

The National Center on Educational Outcomes (NCEO) with the state of South Dakota conducted research that informed the development of Technical Report 66. The authors of this report received assistance from others to plan and implement the research. Authors also received assistance in drafting sections of this report.

For their support, special thanks go to:

  • Diane Halpin, former Research Associate at the Institute on Community Integration at the College of Education and Human Development, University of Minnesota
  • Kristin Liu, Senior Research Fellow at the National Center on Educational Outcomes
  • Rebekah Rieke, Project Coordinator at the National Center on Educational Outcomes
  • Linda Turner, Assessment Supervisor of Special Education Programs for the South Dakota Department of Education


Executive Summary

Accommodations are changes in practices or procedures that facilitate access to instruction and assessment for students with disabilities. Students who struggle to decode written text may benefit from a "read aloud" accommodation (Thurlow, Moen, Lekwa, & Scullin, 2010), which presents text-based material through auditory rather than visual means (Bolt & Roach, 2009). Read-aloud accommodations are used frequently for instruction (Newman, 2006), and assessment purposes (Altman, Cormier, Lazarus, & Thurlow, 2012; Bolt & Thurlow, 2004; Christensen, Braam, Scullin, & Thurlow, 2011). However, evidence to support the use of the read-aloud accommodation on tests for accountability is mixed (Cormier, Altman, Shyyan, & Thurlow, 2010; Thurlow, Lazarus, & Christensen, 2012).

Given inconsistencies in the literature for the read-aloud accommodation, more research is needed to determine how this accommodation is used in practice. Previous research used educators' perceptions to explore the participation of students with disabilities on large-scale tests, including the use of accommodations (see for example, Altman et al., 2012). The current study conducted by the National Center on Educational Outcomes (NCEO), in collaboration with the South Dakota Department of Education, used focus group methodology to look closer at what happens in the room on test day when the read-aloud accommodation is used on mathematics assessments. Focus groups were conducted with educators who had administered the read-aloud accommodation for the South Dakota state mathematics assessment.

Focus group discussions revealed a variety of issues on the read-aloud accommodation. Educators agreed that although it is a frequently used accommodation, its use and effectiveness might vary depending on the content area. Educators also indicated a need for training on how to appropriately and accurately provide the accommodation. For example, educators discussed appropriate expression and intonation during test administration, as well as issues related to the reading of mathematical operations, symbols, and terms.

Other issues identified by focus group participants were practical and logistical factors. Educators spent considerable effort arranging small groups and testing locations for students using the read-aloud accommodation. Some challenges in administration could not be prevented, even with careful preparation. Educators discussed cognitive and social-emotional benefits associated with use of the read-aloud accommodation. For example, increased attention, on-task behavior, and decreased anxiety were generally considered possible benefits.

Results from this study may inform local and state level policies on read-aloud administration. Appropriate selection and administration procedures will ensure that all students will be able to show what they know and can do on assessments.


Overview

Accommodations are intended to promote access for students with disabilities. Students may have physical, sensory, or cognitive challenges that prevent them from showing what they know and can do (Bolt & Roach, 2009). When used for assessment, accommodations remove obstacles "immaterial to what the test is intended to measure" (Thurlow, Lazarus, & Christensen, 2008, p. 17). Accommodations should be based on the student's individual needs (Thurlow, Elliott, & Ysseldyke, 2003), and data should be collected during instruction and assessment to ensure that the accommodation is working as intended (Elliott & Thurlow, 2006).

Title I of the Elementary and Secondary Education Act of 2001 and the Individuals with Disabilities Education Improvement Act of 2004 require that all students, including students with disabilities, participate in state accountability systems. Accommodations play an important role in providing equitable access on state tests for accountability. State policies provide guidance for the types of accommodations that may be used and how accommodations decisions should be made (Thurlow, 2007). In addition, federal regulations released on April 9, 2007 for Alternate Assessments based on Modified Achievement Standards (AA-MAS) highlighted states' role in supporting "the use of appropriate accommodations to increase the number of students who are tested against academic achievement standards for the grade in which a student is enrolled" (Sec. 200.6[a][1][ii][A]).

The number of students with disabilities using accommodations on state tests for accountability varies across U.S. states and has for some time (Altman, Thurlow, & Vang, 2010; Thurlow, Moen, & Altman, 2006). More than half of the states report that 50 to 74 percent of elementary students with disabilities use accommodations in math, and an additional six states report that more than 75 percent of elementary students with disabilities use accommodations in math. However, in two states, fewer than 25 percent of elementary students with disabilities use the read aloud accommodation in math (Altman et al., 2010).

An accommodation frequently allowed on state mathematics tests is items and directions read aloud (Thurlow, Lazarus, Thompson, & Morse, 2005). A "read aloud" accommodation is intended to provide access for students with disabilities who have difficulty decoding written text (Thurlow, Moen, Lekwa, & Scullin, 2010), and may require the use of human readers, audiotapes, or screen readers to present a test through auditory rather than visual means (Bolt & Roach, 2009).

Evidence to support use of the read-aloud accommodation on mathematics assessments is mixed. An analysis of mathematics test scores from the National Association of Educational Progress (NAEP) showed a differential benefit for the read-aloud accommodation when used for students with disabilities compared to students without disabilities (Weston, 2002). Another study compared the test performance of students with and without disabilities on mathematics items with high linguistic complexity, and found that students with disabilities were more likely to benefit from the accommodation (Helwig, Rozek-Tedesco, and Tindal, 2002). Other studies have shown that the read-aloud accommodation may benefit students with and without disabilities. For example, Elbaum (2007) found that accommodated test scores in mathematics increased to a greater extent for students without disabilities than students with disabilities. Given inconsistencies in the research literature, there appears to be limited evidence to guide the use of the read-aloud accommodation in practice (Tindal & Anderson, 2011).

Previous research has used educator perceptions to study the participation of students with disabilities in large-scale assessments (see for example, Altman, Cormier, Lazarus, & Thurlow, 2012). Given mixed empirical evidence for the use of the read-aloud accommodation in mathematics, educators may provide further insight into how the accommodation is used in practice. Educators who administer accommodations during instruction and assessment may clarify and extend how the read-aloud accommodation supports students in demonstrating knowledge and skills.

South Dakota

The National Center on Educational Outcomes (NCEO) with the state of South Dakota explored issues related to the read-aloud accommodation in math on large-scale assessments. South Dakota wanted to learn more about how the accommodation was being used and whether current training was effective.

The state of South Dakota has developed materials to assist educators in providing the read-aloud accommodation on statewide tests. Some of the materials were located in the Test Coordinator's Handbook for the Dakota Test of Educational Progress (DSTEP). The document included guidelines for the administration of accommodations on statewide tests for accountability. One section of the document identified guidelines that apply specifically to staff administering the read aloud. According to the Handbook, administrators should be able to speak clearly with good English pronunciation, be familiar with the vocabulary used in the test, and be able to demonstrate patience and repeat directions or questions. The Handbook requires that "teachers/examiners, teachers/aides, and anyone who participates in the administration of the DSTEP assessment each signs a Test Security Agreement/Affidavit"; this includes test administrators for the read-aloud accommodation (South Dakota Department of Education, 2010).

The Handbook also identified practices that should occur before, during, and after test administration. For example, administrators should be familiar with the terminology used on the test, have reviewed guidelines for reading mathematical expressions, avoid using voice inflection that may influence students' responses, and refrain from discussing test items or responses after testing (South Dakota Department of Education).

South Dakota also provided other training materials on the read-aloud accommodation. For example, the authors obtained a PowerPoint document used for training purposes in South Dakota (see Turner, 2011). One section of the document provided information for staff reading the DSTEP aloud and outlined general guidelines for read-aloud administration. Logistics and scheduling for administration was highlighted. According to the training document, the read-aloud accommodation should be administered to small groups of approximately four students who are in the same grade and have the same test form. Further, the training document indicated that the read aloud cannot be administered in the hallway. The document indicated that administrators should read test items at a pace that is appropriate for an individual student or small group, and should wait until all students in a small group have responded to an item before transitioning to the next item. In addition, the guidelines recommended instructional practices for preparing students for the read aloud. It was recommended that educators teach students to highlight or underline key information when listening to passages or items read aloud, ask that test items be read aloud prior to reading passages, and self-monitor for understanding when listening to passages or items read aloud.

The PowerPoint document also provided specific guidelines for reading items on statewide mathematics tests. Guidelines were intended to prevent the read-aloud accommodation from interfering with the target skill assessed. For items that assessed the student's ability to convert fractions to decimals (or vice versa), test administrators providing the read-aloud accommodation were required to read the stem or answer choices as digits. For example, the numerical expression 1.700 might be read "one point seven zero zero" For items that assessed the student's ability to express numerical expressions in words, the numerical expression was required to be read as digits to prevent the accommodation from interfering with the target skill. For example, the numerical expression 605,788 might be read "six zero five comma seven eight eight" (Turner, 2011).

Purpose

The current study used focus group methodology to explore accommodations practices inside the room on testing day. Focus groups were conducted with educators who had administered the read-aloud accommodation during the mathematics assessment. The current study sought to learn more about test administrators' perceptions of the read-aloud accommodation, with respect to the following questions:

  1. What are appropriate practices in administration of the read-aloud accommodation?
  2. How does the read-aloud accommodation benefit students?
  3. What differences exist in the use of the read-aloud accommodation in math versus other content areas (e.g., reading, science)?
  4. What differences exist between the read-aloud accommodation and other accommodations for students?

Focus Group Procedures

Participants

The South Dakota Department of Education identified school staff who had administered the read-aloud accommodation to students on statewide assessments. The Assessment Supervisor of Special Education Programs for the South Dakota Department of Education selected potential participants from the Western and Southeastern regions of the state. NCEO staff invited educators to participate via e-mail. Educators who responded to the e-mail were included in the study.

Twelve educators participated in the research. Most participants were special educators (n = 10); two participants were directors of special education at the district level. All participants had administered the read-aloud accommodation to students on statewide mathematics assessments. Some had also administered the read-aloud accommodation on statewide tests for other content areas (e.g., reading, science). Focus groups included four participants each.

Setting and Timing

Focus groups occurred in May and June 2011. One focus group was held in May in the Western region, and two focus groups were held in June in the Southeastern region of the state. All focus groups were held at the administrative offices of Local Education Agencies (LEAs). Each focus group was two hours in duration.

Procedures

NCEO staff developed a list of questions for use in each focus group session. The questions aligned with Krueger and Casey's (2000) five categories of questions. Opening questions were intended to make participants feel at ease, and to possibly increase the likelihood that participants would contribute to later discussion. Introductory questions helped identify participants' initial thoughts about the topic. Transition questions probed for more detail and depth in participants' thoughts about the topic, possibly including participants' past experiences. Key questions specified the topics of interest that were most important to the study. Ending questions identified important points discussed and points that may have been missed, and also helped to close the focus group session.

See Table A1 in Appendix A for the focus group protocol. The key questions were most closely related to the research questions. Additional prompts were provided for all key questions. For example, the first key question began: "Explain how you administer the read aloud." Following the key question, prompts were provided to identify practices that should occur before, during, and after a read-aloud administration. Educators also were provided key words (e.g., "Preparation," "Rapport") to assist in identifying appropriate practices. We were interested in learning more about what test administrators considered to be the most important practices in providing the read-aloud accommodation. The other key questions also included several prompts.

Two NCEO staff members facilitated all focus groups sessions. Focus group sessions were audio recorded. Audio recordings were transcribed following the third focus group in June 2011.

Researchers analyzed transcripts for each focus group. Results for the research questions and related themes are reported. Participants' responses for each of the categories (excluding opening questions) were considered for analysis.


Results

Summary tables for each of the four research questions include responses from all of the focus groups.

Research Question 1: What are appropriate practices in administration of the read-aloud accommodation?

Table 1 shows educators' responses to the first research question. The following paragraphs provide examples and more detailed information for each response.

Table 1. Question 1: Summary of Focus Group Responses

Question 1

Focus Group Responses

What are appropriate practices in administration of the read-aloud accommodation?

  • Review and prepare test materials
  • Identify appropriate test setting and schedule in advance
  • Monitor speech quality (e.g., intonation, expression, pronunciation, and pace)
  • Establish rapport with students prior to testing
  • Prepare students for test day and monitor students during testing

Review and Prepare Materials. Participants in all groups said that they reviewed and prepared test materials prior to administering the read-aloud accommodation during assessment. Educators discussed the benefits of reviewing various test materials prior to administration, including test booklets and training guides for test administrators, although the availability of test materials varied among groups, largely due to test security protocols. Some educators received test booklets one day prior to testing, but were required to sign a waiver and keep materials under lock and key when not in use. Other educators did not have access until the morning of the test.

Educators shared that reviewing the test booklet ahead of time was helpful to ensure accuracy and quality of speech. For example, one educator said that looking over the math test helped her to pronounce the items as directed in the states' guidelines, and to avoid unduly influencing students' responses. Educators who did not have access to the student test booklets before test day said that they reviewed training guides for test administrators.

Identify Test Setting and Schedule. Teachers who administered the read-aloud accommodation in math were responsible for scheduling where the students would take the assessment. Two groups discussed some of the difficulties involved in scheduling. For example, educators struggled to find adequate classroom space and personnel for individual or small group administrations. Logistical challenges resulted in testing situations that were not ideal for teachers or students. For example, one educator said that she provided the accommodation to a classroom of students who did and did not have the read-aloud accommodation specified on their IEPs. She walked to each student's desk individually during testing to administer the accommodation, which was cumbersome when administering the accommodation to multiple students. She also said that an aide was sometimes present, but was not consistently available due to scheduling issues.

Administration of the accommodation varied from one-on-one to small group. Educators differed in their opinions of the optimal student/administrator ratio for the accommodation. One educator asserted that one administrator per two students was an optimal ratio for the read-aloud accommodation, while another advocated for a larger group administration. According to one educator, testing was "much more successful in our building than when we had everybody going at the same pace." One educator provided the accommodation to approximately 25 students at one time with assistance from paraprofessionals.

A factor in determining the size of small groups was the number of students who were expected to move through the test at approximately the same rate. Educators said that students who tested at a similar pace would be selected for participation in the same small group. According to one educator, some students were grouped together because they "need the extra time to process."

Monitor Speech Quality. Educators expressed various concerns about speech quality. Self-monitoring for intonation and expression were discussed. Participants said that they monitored themselves as they read to ensure careful intonation, expression, and to avoid unduly influencing students' responses.

All groups discussed challenges related to speech accuracy when administering state mathematics tests. Participants cited mathematical expressions, including numbers, fractions, and operations (e.g., multiplication, division) as the most difficult to pronounce. Educators said that some mathematical expressions required special pronunciations during assessment to prevent invalid inferences about student performance.

However, such practices were criticized. According to one educator, "You don't teach math ‘three comma four five six.' So when you are reading a math test to a student it's basically German to them." Using different pronunciations across instruction and assessment may have deleterious effects for students. One educator said that, "We're pulling the carpet out from under them when we're changing our language."

Educators also discussed potential mistakes during read-aloud administration. One focus group asserted that best practice was to re-read the entire passage or the answer choices to avoid influencing the students and provide them with an accurate reading of the sentences for optimum understanding.

Educators reported that, on occasion, community members or paraprofessionals were responsible for administering the read-aloud accommodation. One focus group expressed concern over the fidelity of the accommodation when these individuals administered the read aloud.

Educators also discussed the importance of monitoring the pace at which they administered the read-aloud accommodation. This involved how fast the administrator reads an item, the punctuation used as he or she reads, and how much time was provided between each item to allow the student to respond. Educators said that pacing was particularly difficult when administering the test in a small group. For example, a group of three students may vary in their individual pace, resulting in confusion for the administrator. To solve the problem, educators assigned students to small groups based on individual pacing, or provided the read aloud to each student in the small group individually. The latter option required considerable effort on the part of the administrator, and may have prevented some students from moving through the test at an appropriate pace.

Establish Rapport with Students. Educators discussed the importance of establishing rapport with students prior to test day. The relationship was beneficial for both the assessment administrator and the students. For administrators, prior rapport with the students helped the educators to pace the read aloud, as well as recognize signs of student frustration. Knowing students ahead of time also facilitated grouping students for the read-aloud accommodation. Educators reported that students were more comfortable with assessment administrators and were more willing to follow the reader when a prior relationship had been established.

Although student-teacher interactions were characterized as warm and encouraging in the classroom, teachers indicated that they refrained from behaviors that might compromise the standard administration of the assessment. Educators reported that they consciously maintained an emotional distance to avoid influencing students during testing, especially when they were familiar with an individual or group of students.

Prepare and Monitor Students. Educators reported that they ensured students were prepared to take the assessments. Educators reported checking that students had taken medications, had an appropriate amount of sleep, and were not hungry. Educators also talked with students to get a sense of how they were feeling, and to provide encouragement about doing their best on the test.

Educators discussed monitoring students once the assessment had commenced, to identify signs of frustration and anxiety; these may assist in the timing of breaks during testing. Educators were also likely to follow each student's progress during the assessment. Some educators provided cues to increase student's ability to attend during testing. For example, one educator used sticky notes to block off sections on the answer sheet, so that only items that the student was working on were exposed. Educators also redirected students if they were off-task or talking. Some students talked to the administrator to verify whether an item had been answered correctly, which educators agreed was inappropriate. One educator said that students were instructed to refrain from asking inappropriate questions to the administrator, including questions that might compromise the validity of the test.

Research Question 2: How does the read-aloud accommodation benefit students?

Table 2 shows educators' responses to the second research question. Explanations and examples are provided.

Table 2. Question 2: Summary of Focus Group Responses

Question 2

Focus Group Responses

How does the read aloud accommodation benefit students?

  • Promotes cognitive engagement (e.g., focus, attention, and motivation)
  • Promotes social-emotional engagement (e.g., reduces anxiety and frustration)
  • Increases student performance

Cognitive Engagement. Focus group participants believed that students received various cognitive benefits from taking the mathematics assessment with a read-aloud accommodation. All three groups asserted that the read-aloud accommodation increased student attention to the assessment. Educators noted that students were more likely to stay on task and follow along when provided the read-aloud accommodation. One group attributed greater attention to the assessment to the physical proximity of the administrator. Another group corroborated the initial increase in attention, but noted that students began to "check out" as the assessment continued.

Social-Emotional Engagement. Educators discussed how the read-aloud accommodation promoted social-emotional engagement through reduced student anxiety, frustration, and confusion; increased motivation; and decreased disruptive behavior.

Educators noted that students' anxiety was reduced because students did not have to struggle to read unfamiliar words. Improvements in students' confidence, posture, and body language were also noted. Overall, students seemed to have more confidence in their abilities because "they're not being tested on their reading." Two other factors related to the read aloud accommodation that appeared to reduce anxiety were: taking the test in a familiar and comfortable environment (rather than large group test setting), and providing timely breaks for students.

Two focus groups said that testing increased stress and anxiety because the test material was not consistent with the material learned during instruction. One educator thought that the accommodation was actually a stress elevator due to the student's close proximity to an adult. To reduce stress, educators reminded students that the test would be difficult, or provided positive reinforcement and encouragement to students. However, one educator thought that increased stress may benefit the student because "without the read aloud they'd probably just fill in the bubbles and be perfectly fine with it."

Two focus groups said that students using the read-aloud accommodation had increased motivation to complete the test, as opposed to testing without the accommodation. One educator said that motivation increased, particularly when the accommodation was delivered in small groups, because "kids could kind of motivate each other to keep working." Another educator compared the use of a human reader to a computer product (i.e., the "test talker"). Without the human interaction, she said most students only casually listened to the recording, and did not appear to exert considerable effort in their responses.

Two groups said that the read aloud helped to decrease disruptive behavior during testing. Because students were closely monitored by adults, students often remained quiet and respectful of other test takers.

Increases Student Performance. According to educators, the read-aloud accommodation in math increased student performance through appropriate pacing of test content and access to the test.

Educators reported that the read-aloud accommodation in math helped students to progress through the test at an appropriate rate. According to one educator, students who received the accommodation were less likely to "rush through it [the test]." However, challenges were identified in pacing content for multiple students. When reading aloud to more than one student, the slower or slowest of the test takers may dictate the speed at which the administrator will read. One educator shared that some students may become bored, noisy, or may ignore the test administrator and proceed at their own pace. Thus, placement in a small group during test administration may serve to decrease a student's focus, even with administration of the read-aloud accommodation. One educator said the read-aloud accommodation reduced "silly mistakes" that students otherwise made due to missing or skipping over words.

One educator shared a story about how tricky it can be to pace students in a small group setting, especially for younger students who struggle with test forms. She said once, in a group administration, a student missed a bubble line and recorded her answer on the line for the question to follow. This continued question after question until she realized the mistake 10 questions later. At this point, the entire group had to stop and wait as the student went back to change her answers.

Access to the test is another benefit of the read-aloud accommodation. One focus group explained that the read-aloud accommodation helped students on mathematics and science tests because students were provided the correct pronunciation for certain vocabulary. Two groups discussed the effect of the read-aloud accommodation on comprehension. According to educators, students would have difficulty comprehending some math terms (e.g., quotient, perpendicular) had they not been provided the accommodation. One educator said, "If they didn't know it [math term], it would cause for more guessing."

Research Question 3: What differences exist in the use of the read-aloud accommodation in math versus other content areas (e.g., reading, science)?

Table 3 shows a summary of educators' responses to the third research question. Examples and details are provided. Results identify findings specific to administration of the read-aloud accommodation in mathematics and reading. Some educators reported similarities between administration of the read aloud in mathematics and science.

Table 3. Question 3: Summary of Focus Group Responses

Question 3

Focus Group Responses

What differences exist in the use of the read-aloud accommodation in math versus other content areas?

  • More preparation and training needed to support the read-aloud accommodation for math assessments
  • Content area knowledge is important for test administrators
  • Reading aloud the test items is perceived more effective for the math assessment than for the reading assessment

Preparation and Training. Most focus group participants indicated that they wanted more time to prepare for read-aloud administration across all content areas. Terminology and administration directions for math were described as particularly difficult. Participants shared that they would like more time to review the tests to ensure appropriate pronunciation of math numbers, terminology, and appropriate explanations for graphs and tables.

According to the educators, more specialized training for the math read-aloud accommodation would be helpful. They noted important differences in how administrators interpreted the directions and how the math items were read to students. One focus group expressed concern that more instructions were needed in the test booklet to specify when items should be read as individual numerals, or as more familiar representations. One participant said, "The consistency seems to be lacking, even from school to school, school district to school district, on how things are presented to the student."

There appeared to be inconsistencies in the provision of training to educators on administering the math assessment. Some educators received training from state department personnel, while others received training from district personnel who had been trained by the state. Others said that they provided informal training to paraprofessionals or other teachers. One educator expressed concern because paraprofessionals or other staff may not have had sufficient training. Educators also discussed the need to collaborate with colleagues. Some indicated that they referred to their peers for assistance, rather than searching the state website or other resource.

Content Area Knowledge. Overall, administrators felt more comfortable administering the read aloud when they had knowledge of the content area. For example, one educator did not feel prepared to give the read aloud in math because she was unfamiliar with the math terminology. She opted to administer the accommodation for the reading test, and asked a colleague to assist with administration for math.

Differences Between the Read-Aloud Accommodation for the Reading Assessment and the Math Assessment. Although not the focus of this study, several focus group participants indicated that the read-aloud directions accommodation on the ELA assessment posed unique challenges for students. Educators said they had to consistently stop at all passages after reading the directions and items and wait for students to read to themselves. Focus groups shared that because students have to read long passages by themselves, students may become overwhelmed, lose focus, and may even begin to guess on the questions instead of reading through the passages. Another shared that the reading test "bogs down" students because the stories are long.

In comparison, educators perceived that administration of the read-aloud accommodation for the mathematics assessment went more smoothly than the read-aloud accommodation for the reading assessment. Test administrators may read everything on the math test, and the items are short. For example, an educator said that the math test was effectively administered using the read aloud "because they're small chunks, they can stay with me and their attention is with me." Given that the reading test seemed to overwhelm students, educators suggested that students who receive the read-aloud accommodation complete the math tests before the reading test.

The focus group participants also discussed considerations for the read-aloud accommodation in instruction. Although reading passages may not be read aloud on the state reading assessment, they are often read aloud for students during instruction. Some students may not have had adequate opportunity to practice reading passages similar to those on the statewide reading test. Further, some reading strategies for struggling learners are not allowed on the assessment (e.g., re-reading the passage, reading the questions before the passage). One participant said, "We teach them skills on how to be good readers and how to read for information, and then this thing doesn't test that at all."

Research Question 4: What differences exist between the read-aloud accommodation and other accommodations for students?

Table 4 shows a summary of educators' responses to the fourth research question. Due to time constraints, one focus group did not discuss Question 4. Support for each of the responses is provided.

Table 4. Question 4: Summary of Focus Group Responses

Question 4

Focus Group Responses

What differences exist between the read-aloud accommodation and other accommodations for students?

  • The read-aloud accommodation is used frequently
  • The read-aloud accommodation is often combined with other accommodations

Used Frequently. Educators indicated that the read-aloud accommodation was more frequently used than other accommodations such as scribe, enlarged print, simplified directions, alternative setting, calculator, and breaks. Generally, educators seemed to favor the read-aloud accommodation as a "safety net" for students. According to one educator, the benefit of including the read-aloud accommodation on students' IEPs is that "it's there in case they [students] want to use it." Although educators appeared to generally favor the read aloud for students, educators acknowledged inconsistencies in the use of the read-aloud accommodation across instruction and assessment. Some suggested that the read aloud may be provided to more students during assessment than instruction.

In addition, the read-aloud accommodation may assist high school students preparing to take year-end examinations or entrance tests. Parents may also advocate for the read aloud more often than other accommodations.

Combined Often. The read-aloud accommodation is often combined with other accommodations, including alternative setting or frequent breaks. Alternative setting typically accompanied the read-aloud accommodation due to use of a human reader (rather than audiotape or screen reader). Some educators were unsure whether alternative setting was marked as an accommodation on a student's IEP, or documented at all, because some assessment administrators assumed that it accompanied the read aloud. Educators discussed some inconsistencies in how multiple accommodations were documented for students at the local versus state level.

Related Themes

Related themes arose from the focus group discussions. Educators addressed eligibility criteria for receiving the read-aloud accommodation in math, the number of students who should receive the accommodation, and how the read-aloud accommodation was used in instruction as well as assessment.

Educators were concerned about selecting accommodations for students. Some educators were confused about the process and criteria to allow a student to have the read-aloud accommodation for math. One focus group questioned whether the accommodation should be given to students with any reading disability, or whether it should be limited to students with specific needs (i.e., decoding, fluency, or comprehension). Further, the read-aloud accommodation was observed to benefit students with behavioral disabilities as well as students with reading disabilities.

Another concern was the use of reading assessment results to determine whether a student would benefit from use of the accommodation on the math assessment. According to some educators, some students were not eligible to receive the accommodation due to high scores on basic reading assessments. However, other educators believed that high-scoring reading students may benefit as much as low-scoring reading students from the use of the read-aloud accommodation on the math test, especially if the student used the read-aloud accommodation on classroom tests. One educator expressed concern about the use of narrow definitions to determine eligibility for accommodations. She said, "I've spent my whole career saying to kids and to parents, ‘we want to find a way to test their knowledge and not to test their disability,' and now it's gotten to the point where it is so tight, it's tough to keep saying that. I don't know if it is actually true anymore."

Other educators expressed concern about older students who were offered the read-aloud option. High school students, in particular, were described as more likely to refuse the accommodation. One educator explained that the read-aloud accommodation was removed from students' IEPs when they repeatedly refused the accommodation, even though it may have benefited students during testing.

English Language Learners. Educators noted special considerations for English Language Learners (ELL) with disabilities who used the read-aloud accommodation, given that ELL students may lack the cultural knowledge and vocabulary necessary for successful performance on statewide tests. Specifically, educators discussed considerations for Native American students receiving the read aloud. One educator commented that from her perspective it was difficult to build rapport with Native American students when administering the test aloud, due to differences between Native American and majority cultures.

Testing Logistics. Providing read-aloud accommodations to students precipitated a number of logistical problems. Additional classrooms were needed for students reading the accommodation. The assessment window created demands on teacher time so that special education students received few services. Providing the read-aloud accommodation for multiple students or groups of students could take most of the school day. When students were not being provided the accommodation, they were also likely to be supported by paraprofessionals. The problem is exacerbated at the high school level because only one grade is tested, leaving three grades of students still needing typical supports.


Discussion

The current study used educators' perceptions to explore the use of the read-aloud accommodation on the South Dakota Test of Educational Progress (DSTEP)-Mathematics. Through the use of focus group methodology, educators identified important issues on the practices, benefits, and use of the read-aloud accommodation. Educators' reports supported results from previous research in other states (i.e., Altman et al., 2012) that the read-aloud accommodation is among the most frequently used.

Some focus group participants seemed to have limited awareness of the read-aloud guidelines in the test coordinators' handbook and related training materials. Others were familiar with the guidelines, but thought that some of the pronunciation rules for mathematical expressions differed from current pronunciation practices used during instruction. Overall, educators' responses showed inconsistencies in how common mathematical expressions were pronounced. Consistent pronunciation is needed to ensure that the test produces valid inferences of what students know and can do.

Educators also expressed considerations for training and preparation. Many of the participants indicated a need for training to promote accuracy and speech quality, including appropriate intonation and expression. One way to promote speech quality is to provide test administrators access to test materials, including test booklets, prior to administering the read aloud. And, the study results indicated that some educators who administered the read-aloud accommodation in math did review the test items prior to administering the test. Although reviewing test items prior to administration may promote speech quality, it appears not to follow current state policies. South Dakota's guidelines, for example, indicate that test booklets may "not be revealed to students or teacher/examiners prior to testing" (South Dakota Department of Education, 2010, p. 11). This study suggests that allowing test administrators to review materials ahead of time may improve the validity of test results, given that test administrators would be less likely to use inappropriate pronunciation and interfere with the target skills assessed. The state may want to consider how to balance test security with good accommodations practices.

Test preparation extended to scheduling and timing. Educators dedicated considerable effort to assign individual students to a large or small group setting. Some students were assigned to groups of students with similar needs. Often, educators were limited by classroom space or staffing issues. Timing was also critical. Educators struggled to pace all students appropriately, especially students in a group setting. Thus, logistical factors played a critical role in how the read aloud was administered. If the test is administered in small groups, the groups for read-aloud administration should be determined well in advance of test day. If possible, groups should be determined based on the students' ability to progress through the test at a similar pace.

Another issue uncovered in the focus groups related to test administrators' perceptions of how students benefited from the accommodation. According to Bolt and Roach (2009), the intended purpose of the read-aloud accommodation is to provide access to the test for students with reading difficulties; however, the educators highlighted the effect of the accommodation on student's ability to attend, focus, and maintain an appropriate pace during testing. Further, the administration of the read aloud accommodation resulted in a decrease in disruptive behavior for some students. The usefulness of the read-aloud accommodation may extend to students with a wide range of needs, including needs related to attention and motivation. Secondary benefits discussed were decreased frustration and increased social-emotional engagement. Social-emotional benefits were noted, including increased student focus and attention, when comparing the effect of a human reader and technology-based methods of delivering the read-aloud accommodation.

The read-aloud accommodation in math is one of the most commonly used accommodations in South Dakota, but—based upon the discussion of the focus group participants—it appeared that the IEP teams across the state had inconsistent understandings about when the accommodation should be selected. IEP teams may need additional training on how to select, implement, and evaluate the read-aloud accommodation. IEP teams may also need training to help them make appropriate decisions about when the read-aloud accommodation should be bundled with "breaks as needed" or other accommodations. For example, students prone to fatigue or frustration may benefit from also having the "breaks as needed" accommodation.

South Dakota state guidelines for read-aloud administration require that test administrators have good English pronunciation, and be familiar with vocabulary or terms used on the test. Test administrators are not required to have a teaching license. Additional guidelines may be needed to help ensure that test administrators have the knowledge and skills needed to appropriately administer this accommodation. The findings of this study suggest that test administrators should have content area knowledge appropriate for the test they will administer. Test administrators who provide the read-aloud accommodation should receive formal training in read-aloud administration. Training should be provided by state department of education employees, or others who have been trained in official procedures for read-aloud administration. In addition, test administrators who are familiar with the student they are testing may be better able to establish an appropriate pace, or know when the student is in need of a break. Precautions should be taken, however, to ensure that test administrators do not engage in practices that would impair the validity of the test (e.g., providing hints or correct answers) for students they know well.

A related issue is that students sometimes are not familiar with the way that numbers and mathematical terms will be read to them during the test. During instruction, students should be exposed to the varied ways that mathematical expressions may be pronounced on test day.

The current study provides a closer look at what happens in the room on test day. Results may inform state and local level policies on the read-aloud accommodation. Improved selection and administration practices will help ensure that students are able to show what they know and can do, and improve the validity of test results.


References

Altman, J. R., Cormier, D. C., Lazarus, S. S., & Thurlow, M. L. (2012). Tennessee special education teacher survey: Training, large-scale testing, and TCAP-MAAS administration (Technical Report 61). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Altman, J. R., Thurlow, M., & Vang, M. (2010). Annual performance report: 2007-2008 state assessment data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Bolt, S. E., & Roach, A. T. (2009). Inclusive assessment and accountability: A guide to accommodations for students with diverse needs. New York, NY: Guilford Press.

Bolt, S. E. & Thurlow, M. L. (2004). Five of the most frequently allowed testing accommodations in state policy. Remedial and Special Education, 25(3)

Christensen, L. L., Braam, M., Scullin, S., & Thurlow, M. L. (2011). 2009 state policies on assessment participation and accommodations for students with disabilities (Synthesis Report 83). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Cormier, D. C., Altman, J. R., Shyyan, V., & Thurlow, M. L. (2010). A summary of the research on the effects of test accommodations: 2007-2008 (Technical Report 56). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Elbaum, B. (2007). Effects of an oral testing accommodation on the mathematics performance of secondary students with and without learning disabilities. The Journal of Special Education, 40, 218-229.

Elliott, J. L., & Thurlow, M. L. (2006). Improving test performance of students with disabilities on district and state assessments (2nd edition). Thousand Oaks, CA: Corwin Press.

Helwig, R., Rozek-Tedesco, M. A., Tindal, G., Heath, B., & Almond, P. J. (1999). Reading as an access to mathematics problem solving on multiple-choice tests for sixth-grade students. The Journal of Educational Research, 93(2), 113-125.

Krueger, R. A., & Casey, M. A. (2000). Focus groups: A practical guide for applied research (3rd edition). Sage Publications, Inc.: Thousand Oaks, CA.

Newman, L. (2006). Facts from NLTS2: General education participation and academic performance of students with learning disabilities. Menlo Park, CA: SRI International. Retrieved November 15, 2011, from: www.nlts2.org/fact_sheets/nlts2_fact_sheet_2006_07.pdf

South Dakota Department of Education (2010). South Dakota Test of Educational Progress: 2010 Test Coordinator Handbook. Retrieved from: http://doe.sd.gov/octa/assessment/documents/SD10_TCH_DOE.pdf

Tindal, G., & Anderson, D. (2011). Validity evidence for making decisions about accommodated and modified large-scale tests. In S. Elliot, R. Kettler, P. Beddow, & A. Kurz (Eds.), Handbook of accessible achievement tests for all students: Bridging the gaps between research, practice, and policy. New York: Springer.

Thurlow, M. L. (2007). State policies and accommodations: Issues and implications. In C. Cahalan-Laitusis & L. Cook (Eds.), Accommodating students with disabilities on state assessments: What works? Arlington, VA: Council for Exceptional Children.

Thurlow, M. L., Elliott, J. L., & Ysseldyke, J. E. (2003). Testing students with disabilities: Practical strategies for complying with district and state requirements. Thousand Oaks, CA: Corwin Press.

Thurlow, M. L., Lazarus, S. S., & Christensen, L. L. (2008). Role of assessment accommodations in accountability. Perspectives on Language and Literacy, 34(4), 17-20.

Thurlow, M. L., Lazarus, S. S., & Christensen, L. L., (2012). Accommodations and modifications for assessment. In Lloyd, J. & Landrum, T. (Eds.), Effective practices for special education. Arlington VA: Council for Exceptional Children/Pearson.

Thurlow, M. L., Lazarus, S. S., Thompson, S. J., & Morse, A. B. (2005). State policies on assessment participation and accommodations for students with disabilities. The Journal of Special Education, 38(4), 232.

Thurlow, M., Moen, R., & Altman, J. (2006). Annual performance reports: 2003-2004 assessment data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., Moen, R. E., Lekwa, A. J., & Scullin, S. B. (2010). Examination of a reading pen as a partial auditory accommodation for reading assessment. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

Turner, L. (March, 2011). Accommodation: Reviewing accommodations and preparing for Dakota STEP testing. Retrieved from http://doe.sd.gov/oats/documents/AccomMeeting/PubData/Engine/Default.htm?http%3A%2F%2Fdoe.sd.gov%2Foats%2Fdocuments%2FAccomMeeting%2FPubData%2F

U.S. Department of Education (2007a, April 9). Final Rule 34 CFR Parts 200 and 300: Title I - Improving the academic achievement of the disadvantaged: Individuals with disabilities education act (IDEA). Federal Register. 72(67), Washington DC: Author.

Weston, T. J. (2002). The validity of oral accommodation in testing. National Assessment of Educational Progress Validity Studies Panel. Retrieved from http://www.nces.ed.gov/pubs2003/200306.pdf


Appendix A. Focus Group Protocol

Table A1. Focus group questions, time allotted for discussion, and question categories as defined in Krueger and Casey (2000)

Time allotted

Categories and Questions

10 minutes

Opening:

1. Tell us who you are, where you have been giving the read aloud test administration, and what you most enjoy doing when you are not working in a school.

15 minutes

Introductory:

2. What is the first thing that comes to mind when you hear the words "read aloud accommodation"?

15 minutes

Transition:

3. Tell us about the first time you administered the read aloud accommodation to a student.

Additional prompts: What was the experience like? What were your first impressions?

4. Tell us about the most recent time you administered the read aloud accommodation to a student. What was the experience like?

60 minutes

Key:

5. Explain how you administer the read aloud.

Additional prompts:
What should happen before, during, or after administering read aloud?
What are best practices?
What should be avoided?
Considering (any of the below), does anything else come to mind?
Preparation
Rapport
Quality of speech
Influence

6. How does the read aloud accommodation benefit students?

Additional prompts:
Considering (any of the below), is there more to add?
Performance
Physical benefits
Additional benefits (e.g., emotional benefits, motivation)

7. Compare the read aloud administration in mathematics and in other content areas.

Additional prompts:
Consider (any of the below), is there more to add?
Reading
Science
Consider (any of the below), what else might you add?
Student benefit
Preparation
Rapport
Quality of speech
Influence

60 minutes (continued)

8. What are the differences between the read aloud accommodation and other accommodations?

Additional prompts:
Consider (any of the below), is there more to add?
Classroom administration
Accommodations categories (e.g., presentation, response)
Non-standard/unacceptable administration

20 minutes

Ending:

9. Suppose you had 1 minute to talk to the state assessment director on the topic of read aloud accommodations for students with disabilities in math. What would you say?

10. Either of the questions below:

Have we missed anything?
Is there anything we should have discussed about the read aloud but didn't?

2 hours total