|
Sheryl S. Lazarus • Martha L. Thurlow • Jason R. Altman • Rebekah Rieke April 2012 All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Lazarus, S. S., Thurlow, M. L., Altman, J. R., & Rieke, R. (2012). An evaluation of the extent to which teachers used the IEP Quality Tutorial-South Dakota after training (Technical Report 63). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Executive SummaryDuring the 2010-2011 school year, the state of South Dakota piloted an online program called the IEP Quality Tutorial-South Dakota (IEPQ-SD). IEPQ-SD was designed to support the implementation of standards-based IEPs in schools throughout the state. Forty-nine educators in South Dakota participated in training on the IEPQ-SD tool, and they were then given access to the tool so that they could use it in their work with IEPs. This report presents the results of an evaluation of the extent to which teachers used IEPQ-SD after training, and their reactions to it. We conducted interviews with five of the training participants approximately six months after the training session to learn about educator perceptions of the training, how their work with IEPs has been affected by the training, and how student experiences have been affected by the training. We also asked questions about what the educators liked about the training and what could be improved for future trainings, and if and how participants thought the IEPQ-SD tool should be rolled out to other educators in South Dakota. Key findings included:
The perceptions of these educators can provide insights into further training on the IEPQ-SD tool as well as implementation of the tool in schools. OverviewIntroductionBoth the Individuals with Disabilities Education Act (IDEA) and the Elementary and Secondary Education Act (ESEA) require that students with disabilities have the opportunity to learn grade-level content. Educators have long struggled to successfully develop and implement Individualized Education Programs (IEPs) for their students with disabilities that are based on expectations that these students will be exposed to and learn grade level curriculum. The practice of using state content standards in the development of IEPs is referred to as standards-based IEPs (Holbrook, 2007). Special educators, in particular, are faced with a unique challenge in incorporating state standards—which may number in the hundreds—into actionable language in student IEPs. They must prioritize, connect, and combine standards as appropriate to ensure that adequate support is provided to students. During the 2010-2011 school year South Dakota piloted an online program called the IEP Quality Tutorial-South Dakota (IEPQ-SD), designed to support the development of, communication about, and implementation of standards-based IEPs. This program was adapted from a similar online program used in Illinois (IEPQ) (Shriner, Trach, & Yell, 2006). Teachers in the South Dakota pilot study attended a training session prior to receiving access to IEPQ-SD. This report presents the results of an evaluation of the extent to which teachers used IEPQ-SD after training and their reactions to it. BackgroundA standards-based IEP has been defined as one in which the IEP team incorporated state content standards in its development (Holbrook, 2007). According to Ahearn (2006), standards-based IEPs create high expectations for students with disabilities and are considered best practice. Standards-based IEPs provide students with better access to academic content, stimulate conversation between special educators and general educators, and can help initiate discussion about schooling using the same curricular language (McLaughlin, Nolet, Rihm, & Henderson, 1999). Several researchers have proposed models or processes for developing standards-based IEPs (see, for example, Holbrook, 2007; Lynch & Adams, 2008; Nolet & McLaughlin, 2005; Shriner, Plotner, & Rose, 2010). Prior to developing IEPs, all IEP team members, including parents, need to be familiar with the general education curriculum, including the state’s academic content standards and state assessments (Holbrook, 2007). There are a large number of standards at any particular grade level. There are also multiple content areas, each with its own set of standards, and students progress to a new set of standards at the beginning of each school year. State-level demand for educators to base all programs for students with disabilities on grade level content standards is uneven across states (Ahearn, 2006). The motivation to ensure that IEPs address content standards was strengthened by U. S. Department of Education (2007) regulations that allowed states to develop an Alternate Assessment based on Modified Academic Achievement Standards (AA-MAS) for students with IEPs who were unable to achieve proficient performance within the year covered by the student’s IEP. The IEP of a student participating in an AA-MAS must include goals that are based on the academic content standards for the grade in which the student is enrolled. The regulations also indicated that the student must have access to the grade-level curriculum. South Dakota ContextSouth Dakota conducted analyses of data to determine whether the state needed to develop an AA-MAS for students who persistently performed poorly on the regular assessment. A group of educators from across the state was convened to learn more about the characteristics of students with disabilities who might be candidates for an AA-MAS and how they received instruction and took assessments. These studies indicated that there was a need to improve the instruction of low performing students with disabilities who might be candidates for an AA-MAS before developing an additional test. One outcome of these studies was that South Dakota decided to develop online training on standards-based IEPs. South Dakota worked with Dr. James Shriner and colleagues at the University of Illinois at Urbana-Champaign to adapt an IEP Quality Tutorial (IEPQ) for South Dakota. The original Tutorial was developed as part of a federal research project (IES grant # R324J060002) and is a Web-based decision-making support system with tools and content based on research of best practices for providing meaningful access to the general curriculum (Shriner, Carty, Trach, Rose, Kim, & Zanton, 2011). The Tutorial focuses on improving the quality of the IEP document and emphasizes that although IEPs must address the academic learning standards of a student’s enrolled grade level, using the standards themselves as annual goal statements is not an appropriate strategy (i.e., annual measurable goals and short-term objectives need to be identified). The IEPQ-SD Tutorial is designed to encourage the use of data to guide the instructional focus documented in the IEP rather than the instructional approaches that will be used. The tool is designed to assist with yearly planning, not the routine teaching plan. Because not all standards are equally important (Ainsworth, 2003), IEPQ-SD incorporates decision-making supports for general curriculum prioritization to target areas for annual goals. The recognition that a student’s needs cut across standards, both within a content area and across content areas, helps teachers decide where to invest available instructional minutes. IEPQ-SD is designed to provide guidance to ensure that a data-based foundation exists for all decisions. Each section of the IEP is supported with decision-making guides regarding what information and factors to include or exclude, and the degree to which the information should influence subsequent decisions. The IEPQ-SD tool includes several components: (a) Help Topics for select sections of the South Dakota IEP, such as present levels of performance, and goals/objectives; (b) Toolbox Resources that include downloadable reference charts and planning sheets for educators, students, and parents to use in IEP development; (c) Goal Assistants (Academic, Functional, and Transition) that help IEP teams with decisions about how to best prioritize State Learning and Social/Emotional Standards for an individual student based on his or her needs and that support the writing process for annual measurable goals and short-term objectives; (d) Case Student Scenarios for four fictionalized students with diverse learning and behavioral needs, and illustrations of all components of a high quality IEP for each student; and (e) Resource Library with evidenced based, best practice references to books, journals, and websites that could assist teams during IEP development. The Pilot StudyThe South Dakota Department of Education initiated a pilot study using the IEPQ-SD tool during the 2010-2011 school year in South Dakota. Following a one-day training, the 49 educators from 11 school districts who participated in the training were given access to the tool so that they could use it in their work with IEPs. The pilot group of educators included special education district supervisors and special educators who worked in inclusive settings as well as those who worked in resource room settings. The one-day training was held in Sioux Falls in September, 2010. A morning presentation promoted the importance of holding students with disabilities to high expectations and of providing a grade-level standards-based curriculum. This was followed by discussion about difficulties in developing and implementing IEPs based on South Dakota’s standards. Educators were then seated in front of computers with wireless Internet access and began a four-hour training session led by the tool developers. Following the training, the educators were given access to the online tutorial and encouraged to incorporate use of the new tool into their work with student IEPs. The purpose of this study was to evaluate the extent to which teachers used IEPQ-SD after training, and their reactions to it. The specific research questions we sought to answer were:
Top of Page | Table of Contents MethodWe conducted structured interviews via telephone with a sample of teachers who attended the IEPQ-SD training session. The interviews occurred at the end of the school year in which teachers received training and initiated use of IEPQ-SD. This format for collecting information allowed us to have a standard set of questions, but also allowed teachers to elaborate on their answers to questions that were most salient to their experiences (Bogdan & Biklen, 1992). We asked teachers a series of questions that we developed in cooperation with James Shriner and South Dakota Department of Education personnel. ParticipantsWe selected a sample of five participants using a roster of educators who attended the training session. Four participants were special education teachers and one was a district special education director. These participants were from the eastern, central, and western regions of the state, and between them worked at the elementary, middle, and high school levels. Two of the special education teachers attended the training alone whereas two attended with at least one other special education colleague from the same school. The district special education director brought several special education teachers from the same school to the training session. We contacted participants through an initial e-mail and a follow-up phone call when necessary. Respondents received a $35 gift card for their participation. ProcessWe conducted interviews with each of the five participants by telephone in March, 2011. We recorded all interviews for transcription and later review. Interviews ranged from 23 minutes to 42 minutes in length. We analyzed interview data for themes using Microsoft Word coding, highlighting, and review features. We merged data from individual interviews in Microsoft Excel by theme and then analyzed them in this software. From these data, we created an Excel sheet structured by theme. Using this sheet, we analyzed and interpreted the data. Counts of statement types and examples of participant responses were marked for later inclusion in written results. We asked the following interview questions in the order listed:
See Appendix A for the complete interview protocol. Top of Page | Table of Contents ResultsStudy results are organized into two major sections. The first presents the results by theme and the second provides an analysis of the comments of infrequent users of IEPQ-SD. ThemesThe three major themes that we identified were: (1) Using the IEPQ-SD; (2) Changes in Student Experiences; and (3) Recommendations for Future Training, Follow-Up Training, and Scale-Up. The findings for each theme are presented in a separate subsection. Participants who spent more time using the IEPQ-SD program had more to say about its possibilities and potential effects than participants who used it less frequently. For that reason, the results section may include disproportionate representation of their viewpoints. In this study, two of the study participants identified themselves as daily or weekly users and offered significantly more commentary than the three study participants who used the IEPQ-SD program less frequently, or in one case, had not used the program since shortly after the fall training in Sioux Falls. Using the IEPQ-SDParticipants spoke about how they have traditionally worked with IEPs throughout the school year and how their work with IEPs has changed since the training. One educator mentioned using the tool to monitor content specific goals and objectives for the current grade for one of her students, as well as looking forward to the things that the student would need to know in the following grade. Participants said that the IEPQ-SD aided them in more effectively writing the IEP. They indicated that it helped them focus on the specifics of the criteria/standards and the connection to the writing of goals. For example, one teacher said IEPQ-SD helped “refine” her writing so that “it could be more understandable to somebody looking at it for the first time.” Another spoke of being motivated to put time into learning how to best use the IEPQ-SD tool because it helped her connect to the standards with “clarity and precise language.” Furthermore, the IEPQ-SD tool helped bridge the gap between the writing of goals for the IEP and carrying out these goals in the classroom. The features of the IEPQ-SD changed the way in which the educators wrote goals and the way in which they communicated these goals to general education colleagues. The study participants believed that this helped those carrying out instruction to understand the underlying standards. The study participants also explained how they used the IEPQ-SD online tool to streamline the information gathering that educators must do in creation and maintenance of the student IEP. At issue, the responses seemed to indicate, is the number of people involved and the communication efforts that are necessary to work with IEPs. An effort must be made to connect with general educators in the school building (many for a student in high school), as well as parents, guidance counselors, and other administrators—and IEPQ-SD made the process easier. The participants said they could synthesize information to share with other colleagues more effectively with IEPQ-SD in contrast with former communication habits that involved asking general educators to sift through entire IEPs. For example, one study participant cut and pasted relevant information into personalized documents for other educators. Several special educators indicated that when they wrote IEPs, general education teachers and other team members had a better understanding of how the standards directly related to goals. For example, one teacher indicated that this new clarity in the writing of the IEP improved expectations of the general education teacher relative to the IEP student. She said, “I think it kind of opened the eyes of the [general education] teachers too, in understanding even though this student is on an IEP, they still need to perform at the test grade standard.” A teacher noted that the parent input form, the student planning sheet, and the standards area of the website were helpful in promoting communication about the IEP. IEPQ-SD enabled her to involve general educators. For example, she said, “They love it because they feel such a larger part of the IEP now, where before it’s like it was the special ed. team that was developing the goals.” Changes in Student ExperiencesParticipants indicated that the IEPQ-SD tool led to positive changes in students’ learning experiences. For example, one educator said that she thought the increased access that students had to their targets and goals resulted in students who were “able to step up to the level of the other students in the class now.” This teacher taught at the high school level and said that her students help her fill out their IEPs. She added that she thought her students were taking more ownership in their learning with the help of the tool:
This teacher also noted that it seemed like her students were doing more of the same things at the same time as their peers without disabilities in the classroom. She highlighted the efforts of one of her students who completed a “senior experience” project—which many students with disabilities had not even been asked to complete in the past. In speaking about that experience, she said, “All of the other students had to write a research paper, he had to write a research paper. He had to do a project, and I held him accountable.” Study participants spoke about the ability of the tool to help students know exactly where they stand in learning some of the key concepts and goals. For example, another teacher said her students were able to more accurately track their progress from beginning on a goal, to 50% attainment, and then those fine improvements that get them over the hump to 80%, 90%, and eventually 100%. She said, “They’ve got an attitude now of, I’ve set a goal to reach and now I’m going to get better.” She spoke of one of her students making so many improvements through active engagement with the IEPQ-SD tool that the student now spends a hundred percent of her time in the general education classroom. The educator said, in talking about her experience working with this student:
Two educators spoke about the ability of the IEPQ-SD online tool to help them build self-advocacy skills with students. Student awareness of what they are learning, compared to what others are learning, and their own needs to get to the goals that they have set for the year was mentioned as something needed to help students make gains throughout the year. One educator said:
Recommendations for Future Training, Follow-up Training, and Scale-upDuring the interviews, researchers asked study participants for their recommendations for rolling out the IEPQ-SD tool or other similar tools. The study participants found the training that they received prior to receiving access to IEPQ-SD to be extremely helpful. Several also thought that it would be useful to have follow-up training to help educators “fine tune their skills.” Participants indicated that the training was well organized and progressed smoothly. Two participants liked how detailed the training was and believed that it was helpful to have time to explore the minute details of the website. Another spoke to the importance of “time to play” so that “the information could stick with us by the time we got back home in front of our own computers.” The same participant was also thankful for the many trainers available to answer questions as educators experimented with the website. All participants concurred that the most effective part of the training was the learning that occurred around writing more effective and efficient IEPs and aligning IEPs with state standards. Most participants pointed to the writing of measurable goals as the most practical concrete take-away, one stating specifically that this is the area that “teachers often struggle with.” One participant focused on the care taken to present educators with an overarching understanding of the importance of writing standards-based IEPs by introducing educators to the new tool, and then helping them understand what it is and why they should be using it. Specifically, participants remembered the initial presentation by Jim Shriner as an effective way to open the training and set the stage for standards-based IEPs, as well as provide a national scope and fresh perspective to this work. One participant mentioned that he found it useful to hear about what other states were doing. Participants believed that the state department of education in South Dakota does a good job, in general, of finding resources and bringing outside people into the state for training when necessary. Two participants stated specifically that they were very happy that the person who built the IEPQ-SD site was there to tell them about it. One commented that she felt that he knew exactly what he was talking about relative to the design and purpose of the site. The single most identified issue was the amount of time it took to learn the full potential of the IEPQ-SD. Two participants indicated that they were so busy that they were unable to find time to explore the tutorial deeply. Participants suggested that it might be helpful to have multiple training sessions and online tools to support educators in IEPQ-SD use. This would enable IEPQ-SD users to develop a deeper understanding over time of the many ways that IEPQ-SD could be used. One district team organized an after school meeting one day among teachers participating in the pilot that the teachers found very helpful. Participants mentioned the importance of administrator support for the tool. For example, one educator spoke about how she received support from her special education director. She saw that collaboration between a team of educators led to increased use by all involved. Another study participant thought that it would be helpful to host an additional formal or organized training—or roundtable discussions—for those who participated in the training. He said he imagined educators would be able to share their experiences with what they found to be the more useful aspects of the tool. All participants supported rolling out the tool to more educators across the state of South Dakota. Reasons Infrequent Users Seldom Used IEPQ-SDThree educators who participated in this study did not regularly use IEPQ-SD. They identified issues that stood in the way of frequent or full use during the pilot study year. One participant, a special education director, said that for his weaker educators, there was so much they were already working on that the administrator felt that he could not ask them to make use of the tool a priority. Another spoke of the resistance to change she saw in some of her colleagues as she tried to collaborate with them using IEPQ-SD. She said she was met with an attitude of, “I’m doing things fine my way so I don’t have to learn something new.” One educator mentioned how success in learning and using the tool could easily depend on how many students are currently on a special educator’s caseload with pending evaluations. She said, “I had a small caseload this year, so most of my IEPs were right at the beginning, right around the training. I used it a lot during those and then I haven’t really used it since before Christmas.” Top of Page | Table of Contents DiscussionStandards-based IEPs can provide “clear links between state standards and specific needs of students, goals, and objectives” (Lynch & Adams, 2008, p. 39). IEPQ-SD is a tool that IEP team members can use to help them develop and implement standards-based IEPs. This study evaluated the extent to which teachers used IEPQ-SD after training, and their reactions to it. The study participants generally found IEPQ to be very useful and believed that teachers across the state would benefit from its use. Different educators have found different avenues to incorporate use of IEPQ-SD into their work, and have done so to a varying extent. Participants valued the importance of developing IEPs based on grade level standards, and indicated that it helped them use more precise language in the IEPs. It also helped special educators and general educators prioritize goals. For those participants who incorporated IEPQ-SD into daily use it became part of teaching for them. For example, one educator stated, “I cannot say how much I like this website. Don’t ever take it away!” Some participants used the tool with their students. When teachers involved students setting their own educational goals, the students appeared to be more motivated to achieve them. There were indications that the tool may have also improved the self-advocacy skills of some students. Study participants indicated that it was important to get buy-in from special education directors and other administrators in their district. Without this buy-in, educators will not have the support necessary to promote frequent use. It is a concern that some administrators may not recognize the importance of getting their staff up to speed on standards-based IEPs. For example, one administrator felt that he needed to instead devote time to other issues that his “weaker teachers” faced. The state may want to consider providing training specifically aimed at administrators to help them better understand the potential of standards-based IEPs. Standard-based IEPs can help teachers plan and implement high-quality instruction for students with disabilities. In addition, IEPQ-SD can provide support for this important endeavor. Top of Page | Table of Contents ReferencesAhearn, E. (2006). Standards-based IEPs: Implementation in selected states. Alexandria, VA: Project Forum, National Association of State Directors of Special Education (NASDSE). Available at: http://projectforum.org/docs/Standards-BasedIEPs-ImplementationinSelectedStates.pdf Ainsworth, L. (2003). Power standards: Identifying the standards that matter the most. Englewood, CO: Advanced Learning Press. Bogdan, R., & Biklen, S. K. (1992). Qualitative research for education. Boston, MA: Allyn and Bacon. Holbrook, M. D. (2007). Standards-based individualized education program examples. Alexandria, VA: Project Forum, National Association of State Directors of Special Education (NASDSE). Available at: http://www.projectforum.org/docs/Standards-BasedIEPExamples.pdf Lynch, S., & Adams, P. (2008). Developing standards-based individualized education program objectives for students with significant needs. Teaching Exceptional Children, 40(3), 36-39. McLaughlin, M., Nolet, V., Rhim, L. M., & Henderson, K. (1999). Integrating standards: Including all students. Teaching Exceptional Children, 31(3), 66–71. Nolet, V., & McLaughlin, M. J. (2005). Accessing the general curriculum: Including students with disabilities in standards-based reform. Thousand Oakes CA: Corwin Press. Shriner, J. G., Carty, S. J., Trach, J., Rose, C. A., Kim, M., & Zanton, J. (April, 2011). Project IEP-Q: A system for decisions on standards-based IEPs. Paper presented at the annual meeting of the Council for Exceptional Children, Washington, DC. Shriner, J. G., Plotner, A., & Rose, C. A. (2010). Development of individual education programs for students with emotional or behavioral disorders: Coordination with transition plans. In D. Cheney (Ed.). Transition of secondary students with emotional or behavioral disorders: Current approaches for positive outcomes (2nd ed.) (pp. 171-214). Champaign, IL: Research Press. Shriner, J. G., Trach, J. S., & Yell, M. L. (2006). The IEP Quality Improvement Project: Research and development of web-based decision support. (IES Grant No. R324J060002). Urbana, IL: University of Illinois. U.S. Department of Education (2007, April 9). Final Rule 34 CFR Parts 200 and 300: Title I- Improving the academic achievement of the disadvantaged; Individuals with Disabilities Education act (IDEA). Federal Register, 72(67), 17748-17781. Top of Page | Table of Contents Appendix AProtocol: Teacher Structured InterviewsEvaluation of the IEPQ-SD (IEP Quality Tutorial-South Dakota) Training Session and Tool: Teacher Structured Interview Guided Questions (prompts listed below question) Thank you for taking the time to complete this interview today. We are excited for the opportunity to learn more about participant perspectives regarding the South Dakota state Department of Education training session that you attended in September on Standards-based IEPs and how you are using the IEPQ-SD Tutorial. Before beginning, please confirm that you have read and understand the consent form that you signed. Be assured that your comments will never be connected to your name or school district. Results will only be used to help improve future trainings, so feel free to be honest and descriptive in your answers. Training Session Questions:
|
