View or download the full report as a PDF document.

Who Are the Students Who May Qualify for an Alternate Assessment Based on Modified Academic Achievement Standards (AA-MAS)?: Focus Group Results 

Synthesis Report 79

January 2011

All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Berndt, S., Ebben, B., Kubinski, E., Sim, G., Liu, K., Lazarus, S., Thurlow, M., & Christian, E. (2011). Who are the students who may qualify for an alternate assessment based on modified academic achievement standards (AA-MAS)?: Focus group results (Synthesis Report 79). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Table of Contents

• Executive Summary
• Background
• Focus Group Procedures
• Results
  • Question 1: Who are the students with disabilities likely to be identified to take an alternate assessment based on modified academic achievement standards?
  • Question 2: What kind of evidence should be used to identify these students?
  • Question 3: What would be the teacher’s role in identifying these students?
  • Question 4: What must the IEP of these students address?
• Conclusions
• References

Executive Summary

Beginning in 2007, federal regulations to two major education laws gave state departments of education the option to develop an alternate assessment based on modified achievement standards (AA-MAS) for some students with disabilities. The regulations stated that the AA-MAS was intended for students who were being instructed in the grade-level curriculum but who were not likely to achieve grade-level proficiency in that curriculum during the year of their Individualized Education Program. States interested in pursuing the development of a test have had to grapple with determining more specific test eligibility criteria. At the heart of AA-MAS eligibility decisions is the question of whether or not students with disabilities have appropriate access to the grade-level curriculum. If students are not given appropriate access to the curriculum, an AA-MAS will not provide a different picture of students’ knowledge and skills than the regular state content assessment.

This report summarizes the results of educator focus groups conducted by one state in a consortia dedicated to studying AA-MAS eligibility issues. In 2008 and 2009, the Wisconsin Department of Public Instruction held three focus groups. In each session, facilitators followed a two-part procedure developed by Berndt and Ebben (2008). First, participants responded to a series of questions about characteristics of students who would be eligible for an AA-MAS, the types of information needed to determine a student’s eligibility, and teachers’ roles in the decision-making process. Second, small groups of participants examined an AA-MAS fact sheet and clarified their understanding of the language found in federal regulations. Through this process, participants addressed commonly held misperceptions that at-risk students without disabilities could be included in an AA-MAS. In addition, they recognized that students with disabilities needed to be instructed in the grade-level curriculum instead of content from lower grade levels. Finally, participants acknowledged the need for standards-based IEPs to clearly address accommodations and modifications used in instruction as well as in assessments.

Based on the results of these focus groups, the Wisconsin Department of Public Instruction has planned further professional development opportunities for both special education and general education teachers.

Top of page | Table of Contents

Background

In 2007, a regulation to both the Elementary and Secondary Education Act of 2001 (also known as the “No Child Left Behind Act”) and the Individuals with Disabilities Education Act of 2004 introduced the possibility of a new kind of statewide accountability assessment. The regulation provided an option for states to develop an alternate assessment based on modified academic achievement standards (AA-MAS). The AA-MAS is in addition to the alternate assessment based on alternate academic achievement standards (AA-AAS) for students with significant cognitive disabilities. The AA-MAS is optional and states are not required to offer it.

According to federal regulations, the AA-MAS is intended for a small group of students with disabilities who can make significant progress in the grade-level, standards-based content, but who are unlikely to reach grade-level proficiency within the time period covered by their Individualized Education Program (IEP). These assessments are based on the same grade-level academic content standards as a state’s regular test, but the test may be less difficult than the regular test. Students who participate in the AA-MAS must have an IEP, and may be from any disability category. States may count up to 2% of all students as proficient for accountability purposes using the AA-MAS (U.S. Department of Education, 2007). According to Berndt and Ebben (2008):

This new alternate assessment option, perhaps more than others, is one that should prompt states to think about who the students are who might take this assessment. Included in the considerations about who the students are might be questions about how the students gain access to grade-level content standards and how they show what they know and are able to do (p. 1).

During the 2009-2010 school year 13 states had an assessment they considered to be an AA-MAS. For states with an AA-MAS the most frequently made design changes from their general assessment were: reducing the complexity of the test language, reducing the number of distracters in multiple choice test items, providing shorter reading passages, having fewer reading passages, using a larger font size, having fewer items per page, having fewer test items overall, and bolding or underlining key text (Hodgson, Lazarus, & Thurlow, in press).

In 2008, the Wisconsin Department of Public Instruction joined a consortium of state departments of education and the National Center on Educational Outcomes (NCEO). The intent of this consortium is to better understand which students might participate in an AA-MAS if a state chooses to implement one. When the AA-MAS first became an option for states, Wisconsin educators had a strong interest in developing such a test, but the state has decided not to do so. Instead, the Department of Public Instruction is interested in exploring ways to improve instructional practices for students with disabilities who may not be achieving proficiency on the general state test, yet who are not eligible to take Wisconsin’s alternate assessment based on alternate achievement standards (AA-AAS), the Wisconsin Alternate Assessment for Students with Disabilities (WAA-SwD).

To that end, the Wisconsin Department of Public Instruction held focus groups with Wisconsin educators to accomplish three specific goals:

1. To help educators become familiar with federal regulatory language about students eligible for the AA-MAS;
2. To familiarize educators with issues that must be considered when determining which students might qualify for the AA-MAS; and
3. To help educators identify strategies for improving instruction and assessment practices for struggling learners.

Top of page | Table of Contents

Focus Group Procedures

Participants

One-day focus groups occurred on three separate occasions over a nine-month period from July 2008 through March 2009. The Department of Public Instruction made an attempt to involve a diverse group of educators, including special educators, general educators, special education administrators, college-level teacher trainers, and parent liaisons as part of each focus group session. Participants did not need to have prior knowledge of the regulations authorizing the AA-MAS. Participants included a roughly representative sample of 45 educators (15 per group) from different geographic areas of Wisconsin.

Timing of the Focus Groups

Group 1 took place in July, 2008. It was held in Madison and included educators from the South Central Region of the state. Group 2 included educators from the Northern Region and took place in Eau Claire in December, 2008. Group 3 was held in Madison in March, 2009 and included educators from the Eastern Region. The first and second focus groups occurred when educators had not had as much exposure to information about the AA-MAS. Listening to the discussion in groups 1 and 2 allowed Department of Public Instruction staff to determine the educators’ general level of understanding about the AA-MAS, and more specifically to determine participants’ understanding of eligibility requirements. These discussions also allowed Department of Public Instruction staff to gauge educators’ professional development needs on the topic of grade-level standards-based instruction for struggling students. In contrast, by the time the third focus group met in late winter of 2009, some new teacher training on standards-based IEPs had already been conducted both independent of, but also directly as a result of the earlier focus group findings. As a result, group 3 participants may have had more prior exposure to the ideas discussed in the focus groups.

Procedures

Focus group facilitators from the Department of Public Instruction followed the procedures outlined in a focus group tool developed specifically for this purpose by Berndt and Ebben (2008). During Activity 1, the facilitator asked each group to respond in turn to four discussion questions:

Question 1: Who are the students with disabilities likely to be identified to take an alternative assessment based on modified academic achievement standards?

Question 2: What kind of evidence should be used to identify these students?

Question 3: What would be the teacher’s role in identifying these students?

Question 4: What must the IEP of these students address?

Teachers responded based on their incidental personal knowledge of AA-MAS requirements. Answers to the discussion questions were recorded and shared with the group.

During Activity 2, participants divided into small groups of three or four people. The small groups studied an AA-MAS fact sheet (National Center on Educational Outcomes, 2007) containing language from the federal regulations and additional clarifying information. Each group revised its original answers based upon information covered in the fact sheets. Examining and discussing the fact sheets allowed small groups to clarify their understanding of AA-MAS requirements. Finally, the participants came together as a large group to discuss two broad questions:

1. What have you learned about the group of students who may qualify to participate in the AA-MAS?
2. Did anything surprise you?

Top of page | Table of Contents

Results

This section of the paper compares and contrasts each group’s initial responses to questions posed during Activity 1 to their later understanding of AA-MAS requirements expressed during Activity 2. Portions of the relevant AA-MAS regulations are included with each question to illustrate what participants were reading. Summary tables for each question show responses from all groups.

Question 1: Who are the students with disabilities likely to be identified to take an alternate assessment based on modified academic achievement standards?

Table 1 shows a summary across groups of answers to question 1. These answers are discussed in more detail in the paragraphs that follow.

Table 1. Question 1: Summary of Focus Group Responses

Activity 1 Responses for Question 1

Initially, Group 1 participants responded to this question with many broad answers. The educators identified several types of students they believed would be eligible for the AA-MAS. The group thought eligible students included any student who scored in the Basic or Minimal range on the general state assessment (Wisconsin Knowledge and Concepts Examination), English Language Learners, general education students achieving significantly below peers, and students with self-esteem, motivation, and attendance issues. When considering students with disabilities, Group 1 did not include students with all types of disabilities as potential participants in the AA-MAS. They omitted students who are deaf or hard of hearing, students with vision impairments, and students with other health impairments. These students represent three of the 13 federal disability categories. In a related discussion not directly connected to eligibility, the group did not identify a need to provide appropriate assessment accommodations.

When asked about characteristics of students eligible for the AA-MAS, Group 2 started by identifying factors related to student performance. The group identified students who were proficient or advanced on the alternate assessment based on alternate achievement standards (the WAA-SwD), students who required multiple accommodations on the general state assessment, or students whose academic goals were at least two years below grade level on their IEP. This group also included students displaying some non-academic student characteristics such as a lack of sleep, hunger, emotional needs, and a lack of confidence. When addressing the participation of students with disabilities, the group only named students with specific learning disabilities and students with emotional behavioral disabilities as eligible.

Group 3 created an initial list of potential participants that was quite broad. The list of participants included students with disabilities. However, the group only accounted for students from five of the thirteen federal disability categories: emotional behavior disability, autism, cognitive disability, learning disability, and other health impairments. They also included students without IEPs such as English Language Learners, students with Section 504 plans, students diagnosed with Attention Deficit Hyperactivity Disorder, students who were low-level readers, students with attendance problems, and students with health issues that caused them to miss school. Group 3 did include students with disabilities who require accommodations, but they placed a low priority on these students and only recognized particular accommodations like teacher defined “read alouds,” students reading aloud, and provision of extra time. According to group 3, other students who might be eligible for the AA-MAS were those who could complete the WAA-SwD easily yet would not be proficient on the general state assessment.

Activity 2 Responses for Question 1

The fact sheet (National Center on Educational Outcomes, 2007) shown in Figure 1 contains federal regulatory language about identifying students who might participate in an AA-MAS.

Figure 1. The Regulatory Language (April 9, 2007) for Identifying Students Eligible for Assessments Based on Modified Academic Achievement Standards (Fact Sheet: Question 1)

After reviewing the fact sheet, Group 1 narrowed the field of who would be eligible to participate in AA-MAS. This group excluded students without disabilities who were achieving as expected and added students in the three omitted disability categories. Last, the group recognized the importance of providing assessment accommodations for students with disabilities who need them.

Group 2 changed its previous response, which had indicated that only students with learning disabilities and emotional behavioral disabilities would be eligible, to include students with all 13 types of disabilities that are found in federal legislation. Group 2 participants affirmed the idea that examining student performance would help identify the students eligible to participate in an AA-MAS. They recognized that external factors sometimes impact how a student performs on a test, and that accommodations are necessary for certain students. One of the most important realizations the group came to was the potential impact of effective access to the general education curriculum on a student’s potential success and proficiency. Group 2 realized that some of their students might not be receiving the instruction necessary for proficiency on the state assessment.

Group 3 participants expanded their previous list of eligible students with disabilities to include students from all 13 federal disability categories. After reviewing the fact sheet, this group only included special education students as eligible for the test, as specified in the federal regulations.

Question 2: What kind of evidence should be used to identify these students?

Table 2 shows a summary across groups of answers to question 2. These answers are discussed in more detail in the paragraphs that follow.

Table 2. Question 2: Summary of Focus Group Responses

Activity 1 Responses for Question 2

Initially Group 1 generated a list of a wide variety of evidence that could be used to identify students eligible for the AA-MAS. Some of the evidence (e.g., poor standardized test performance, poor performance on progress monitoring assessments) was directly related to student achievement and some was indirectly related (e.g., attendance, motivation, and low socioeconomic status).

Group 2 generated lists of potential evidence that focused on the student rather than the test. The majority of evidence this group cited related to academic skills and indicators such as reading skill, writing skill, assessment scores, and instructional level. Other evidence the group named, including test anxiety and attendance, was indirectly associated with academic performance.

Group 3 generated a wide variety of types of general evidence to consider when making AA-MAS eligibility decisions. These types included non-academic evidence (e.g., attendance records, the existence of social/behavioral issues, foster care status, adoption status, living conditions, and length of time in the district). They also included direct academic evidence such as scores on state assessments, results of diagnostic testing or benchmark assessments, teacher observations, and a documented need for accommodations.

Activity 2 Responses for Question 2

Focus group participants read the fact sheet shown in Figure 2.

Figure 2. The Regulatory Language (April 9, 2007) for Identifying Students Eligible for Assessments Based on Modified Academic Achievement Standards (Fact Sheet: Question 2)

Once Group 1 examined the fact sheet, the group members again narrowed their focus. They recognized the need to consider academic factors instead of other non-academic student characteristics. They also recognized that educators could use valid teacher-made assessments as measures of academic achievement. Another area of subsequent discussion was the value of teacher and parent input regarding data collection and analysis. However, Group 1 understood that sometimes there might be insufficient data to make proper educational decisions about a student’s need for AA-MAS.

Group 2 revised its original list of possible evidence, emphasizing the use of academic evidence and test results rather than non-academic evidence. These group members recognized the importance of grade-level academic goals in a student’s IEP. Again, this realization came as group members began to understand the importance of providing students access to the general education curriculum.

As Group 3 reviewed the fact sheet, the members discovered the importance of only examining direct academic factors to make a decision about AA-MAS participation. The participants realized the value of examining the environment, curriculum, and daily schedule for cues on identifying students, but not using them as deciding factors.

Question 3: What would be the teacher’s role in identifying these students?

Table 3 shows a summary across groups of answers to question 3. These answers are discussed in more detail in the paragraphs that follow.

Table 3. Question 3: Summary of Focus Group Responses

Activity 1 Responses to Question 3

Prior to examination of the fact sheet, Group 1 listed many tasks that teachers could perform to facilitate the identification of AA-MAS participants. These tasks included collecting data, observing the student, and selecting appropriate accommodations. The group suggested that teachers could better prepare students with disabilities for tests. For example, teachers could focus on providing test preparation, teaching about different test formats and question types, and encouraging self-advocacy. The group recognized that teachers need to know how to properly use and analyze the data collected and how to incorporate structured classroom observation to accomplish student academic movement.

Group 2 specified from the start that the teacher’s role in identifying students eligible to take the AA-MAS should be data oriented. It thought the teacher needed to perform tasks such as administering running records, looking at longitudinal student data, making data-based decisions, and examining student progress relative to academic benchmarks and state assessments. The educators were also clear in stating that both general education and special education teachers should be involved in the identification of eligible students. They believed that a participation decision should involve everyone on the student’s IEP team.

Group 3 began by listing numerous duties teachers perform to monitor students’ learning. These duties include collecting work samples, documenting progress, issuing report cards, participating in IEP development, monitoring progress in the core curriculum, and reviewing the student’s history. In addition, the group mentioned the importance of teacher collaboration and team decision-making skills combined with an in-depth knowledge of the students. The group had extensive discussion regarding the importance of common professional development for both special education and general education teachers.

Activity 2 Responses to Question 3

The fact sheet (National Center on Educational Outcomes, 2007) shown in Figure 3 contained language about the teacher’s role in student identification.

Figure 3. The Regulatory Language (April 9, 2007) for Identifying Students Eligible for Assessments Based on Modified Academic Achievement Standards (Fact Sheet: Question 3)

Following review of the fact sheet, Group 1 shifted its attention from the teacher’s role in test preparation to the teacher’s role in instruction. It recognized the need for students with disabilities to have access to grade-level instruction in order to be successful, and the need to develop academic goals based on grade-level standards. The group members understood why teachers should expect students to move up in the curriculum. They also prioritized the use of data to inform academic decisions.

Group 2 realized that many of the members’ original ideas about the teacher’s role in data collection and analysis were correct, but could only work with professional development for both general and special education teachers. The group also thought common assessments would make for a more standardized approach to student identification. Group 2 stressed that IEP goals should be aligned with grade-level academic content standards and that making sure students had access to the general curriculum at grade-level was essential to any possible success.

Group 3 members realized they did not previously address key factors such as ensuring students had access to the general curriculum, obtaining parent input, or using and analyzing data to make educational decisions. This group came to the conclusion that the teacher’s role should be data driven and that it is important to obtain data on students’ grade-level instructional outcomes. The group discussed areas for further training such as implementing new instructional strategies, providing scaffolding for students, and striving for a more evidence based approach to instruction and decision-making.

Question 4: What must the IEP of these students address?

Table 4 shows a summary across groups of answers to question 4. These answers are discussed in more detail in the paragraphs that follow.

Table 4. Question 4: Summary of Focus Group Responses

Activity 1 Responses to Question 4

Group 1 generated a list of items to include in the IEP that emphasized the importance of assessment accommodations. The group members also stressed the need to provide instructional accommodations that may not be allowed on the general state assessment. They suggested including information about behavioral/social issues that impact a student on a regular basis. Notably, the group did not address documenting access to grade-level curriculum and instruction.

Group 2 suggested a number of items to document in the IEP, including: (a) the number of minutes the student would be participating and receiving instruction in the general education curriculum, (b) the actual student grade-level performance, (c) the results of curriculum-based assessments, (d) needed accommodations and modifications, (e) types of instructional scaffolding the student may need, and (f) the types and amounts of supplementary aids and services.

Group 3 generated a list of many academic and non-academic factors to address in the IEP. The academic factors and characteristics they suggested were: (a) student reading level, (b) student access to core academics, (c) accommodations needed, (d) assistive technology needed, (e) test score history, (f) student strengths and weaknesses, and (f) previous interventions. Group members also suggested that IEPs should address non-academic factors like attendance, expectations, student attendance at IEP meetings, and length of time in special education.

Activity 2 Responses to Question 4

Participants examined the fact sheet (NCEO, 2007) shown in Figure 4 that contains language about IEPs.

Figure 4. The Regulatory Language (April 9, 2007) for Identifying Students Eligible for Assessments Based on Modified Academic Achievement Standards (Fact Sheet: Question 4)

Group 1 had not previously mentioned access to grade-level instruction in its original list of IEP elements. After reviewing the fact sheet, the members recognized and reinforced the need for including access to grade-level instruction in the student’s IEP. These educators indicated the importance of writing goals that might anticipate student movement from participating in the AA-MAS to taking the general state assessment. They stated that the IEP goals should not indicate that students will stay in an alternate assessment for extended periods of time if the students can move on to the general assessment.

Group 2 indicated an understanding of what teachers and students need to accomplish relative to grade-level instruction. The group became aware of the legal regulations requiring that progress monitoring must include a variety of scientifically-based measures. It also recognized the ways in which data and documentation aid in writing required IEP statements about a student’s present level of functioning. Using these kinds of data leads to the development of academic goals reflecting high expectations.

Group 3 realized it was too broad in its original answers about IEP content and narrowed its responses to this question. For example, they recognized the importance of making participation decisions based on academic factors rather than non-academic factors. The group understood that the IEP needs to focus on needed accommodations and instructional modifications. They also emphasized the importance of addressing student access to the grade-level curriculum.

Top of page | Table of Contents

Conclusions

By using the fact sheet (National Center on Educational Outcomes, 2007) participants independently discovered three key concepts on their own during the focus groups. First, they clarified some commonly held misperceptions about students who would be eligible for the AA-MAS. Before examining the fact sheet many participants believed all types of struggling students would be eligible for the test, regardless of whether the students had a disability or not. After reviewing the fact sheet, however, the participants clearly understood that the AA-MAS option was only for a small group of students with disabilities and that eligibility should be based on academic outcomes rather than on non-academic characteristics or labels.

Second, the participants realized the important role that access to grade-level content instruction plays in determining which students are likely to benefit from an AA-MAS. In the past, the general trend among educators had been to focus on instructing students with disabilities and other struggling students at their developmental and current academic level, often focusing on remediation of the students’ academic skills rather than providing access to grade-level curriculum. As a result, students may have been learning content from lower grade levels. For that reason, identifying students who would benefit from participating in the AA-MAS requires some careful study to differentiate students who have simply not had the opportunity to learn the grade-level content material from those who have had the opportunity to learn the material but have not made sufficient progress during the year covered by their IEP. The group realized that teachers need to bring as many possible sources of evidence as they can to the decision-making process. Using evidence will help AA-MAS decisions to be evidence-based and data-driven.

Finally, the focus groups came to the conclusion that the IEPs of potential AA-MAS participants need to clearly address both classroom accommodations and modifications and assessment accommodations. All groups identified that the intent of classroom and assessment accommodations is to ensure that students are getting the grade-level content in such a way as to meet their disability related needs; also, to ensure they can demonstrate that learning without impediment by their disability. These three major discoveries extend beyond determining who is eligible for an AA-MAS and reach out into improving daily instruction for all students in the grade-level content classroom.

Next Steps

As a result of the information obtained from the three focus groups, the Wisconsin Department of Public Instruction was able to identify areas of need for further professional development and training for both special education and general education teachers. These areas include:

• Access to the general education curriculum for students with disabilities.
• Grade-level instruction for students with disabilities.
• The use of accommodations in instruction and assessment.
• Collecting and using data on the academic progress of students with disabilities.

It is anticipated that students in Wisconsin will benefit as the Department of Public Instruction continues to address these needs through training and professional development.

Top of page | Table of Contents

References

Berndt, S., & Ebben, B. (2008). Thinking about the students who may qualify to participate in an alternate assessment based on modified academic achievement standards (AA-MAS): A tool for focus groups. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved from http://www.cehd.umn.edu/nceo/OnlinePubs/AA-MAStool.pdf

Hodgson, J. R., Lazarus, S. S., & Thurlow, M. L. (2010). Characteristics of states’ alternate assessments based on modified academic achievement standards in 2009-2010 (Synthesis Report 81). Minneapolis MN: University of Minnesota, National Center on Educational Outcomes.

National Center on Educational Outcomes. (2007). Identifying students with disabilities who are eligible to take an alternate assessment based on modified academic achievement standards. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved from http://www.cehd.umn.edu/nceo/Teleconferences/AAMASteleconferences/AAMASIdentifying%20Students.pdf

U.S. Department of Education (2007, May 20). Final regulations on modified academic achievement standards. Washington, DC: U.S. Department of Education. Retrieved from http://www2.ed.gov/policy/speced/guid/modachieve-summary.html