Policy Directions 19

Aligning Alternate Assessments to Grade Level Content Standards: Issues and Considerations for Alternates Based on Alternate Achievement Standards

Shawnee Y. Wakeman • Claudia Flowers • Diane Browder
University of North Carolina at Charlotte

November 2007

Wakeman, S., Flowers, C., & Browder, D. (2007). Aligning alternate assessments to grade level content standards: Issues and considerations for alternates based on alternate achievement standards (Policy Directions 19). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://cehd.umn.edu/NCEO/OnlinePubs/Policy19/

Contents

Background
Issues Complicate Alignment
Current Models of Alignment
Alignment Study Components
Summary
Resources

Background

Technical adequacy of state alternate assessments is now a focus of Peer Review initiated by the U.S. Department of Education. Each state must document the degree to which the alternate assessment based on alternate achievement standards (AA-AAS) aligns with its content standards.

Alignment issues for general grade level assessments have been addressed in many ways. Still, the complex alignment models that are used to conduct alignment studies are relatively new to the field of education. Although multiple options now exist for alignment studies, states are encouraged to consider all elements that can impact the alignment of an AA-AAS assessment system. These include policies and student characteristics, as well as standards and assessments.

The purpose of this Policy Directions is to provide states with information on issues that complicate alignment of alternate assessments based on alternate achievement standards. It also provides information on existing alignment models that can be used for alignment studies.

A companion Policy Directions provides states with components to consider when planning an alignment study (with an external vendor), including guidance for maximizing resources spent to determine the alignment of the state’s AA-AAS with its grade-level content standards.

Issues Complicate Alignment

There are at least three issues that can complicate the development of alignment studies of alternate assessments. The first issue reflects the unique formats used for these assessments, which have been developed to be responsive to the characteristics of students with significant cognitive disabilities. Alternate assessment formats include checklists, performance-based assessments, portfolios, and variations of these alternatives. Each type of format may have a varying degree of standardization. Studying the alignment of portfolios with teacher selected assessment items is far different from considering that of a performance-based assessment with standard items and administration directions.

A second issue is that certain assumptions cannot be made about the educational context. The state cannot assume that students have received instruction on state standards given the lack of emphasis on academic instruction for this population until recently. Teachers may not have received pre-service training on how to teach to state standards. Even teachers well trained in academic interventions may struggle with balancing time spent on teaching academic standards with that devoted to addressing students’ functional life skill needs.

A third issue is that the development of alternate assessment items from state standards is a complex task. While any alignment study may reveal that some assessment items do not align with state standards, for alternate assessments some items may not even be academic. For example, in trying to stretch a standard for students with minimal symbolic communication, alternate assessments may have inappropriately included items like "washes hands" with the good intention of assessing that the student can "demonstrate knowledge of microorganisms in human disease." In applying existing alignment models, states are urged to consider what additional components may need to be evaluated to ensure a full picture of alignment of the alternate assessment is obtained.

Current Models of Alignment

Most alignment studies focus on horizontal alignment—the degree of overlap among academic content standards and assessments. While states may consider additional dimensions in studying the alignment of their general assessments, these additions increase in importance for alternate assessment systems because of the previously described issues.

Figure 1 illustrates a path between some of the educational components that are included in AA-AAS systems, including extended standards, an additional component not found in general education assessment systems. Prior to developing the alternate assessment, many states choose to articulate the prioritization or transformation of these standards to clarify how they will differ in complexity from grade level achievement. This transformation may have a variety of names, such as curricular frameworks, extended standards, and targets for learning. These extended standards should not be different content standards. Instead, they help to clarify how to promote access to the general curriculum for this population. They are the mechanism that states use to identify content targets for alternate assessments.

Figure 1. Paths Between the Educational Components of an Alternate Assessment System

At a minimum, the additional component to be considered in an alignment study of alternate assessments is the relationship between any extended standards used in the development of the alternate assessment and the original standards (Path A) as well as their alignment with the alternate assessment (Path B). These standards also should direct instruction (Path C) as well as assessment, and instruction should be provided on the content to be assessed (Path D). The characteristics of students with significant cognitive disabilities and current state and federal policy also need to be considered in the description of the paths.

The model shown in Figure 1 is a system alignment (all paths) compared with the pared down focus of horizontal alignment between standards and assessment (Path A only). System alignment looks at the linear progression from policy elements to classroom instructional practices to student outcomes. In turn, student outcomes should also inform policy initiatives and revisions resulting in a cyclical effort of reform. Figure 2 represents a straightforward description of this process.

Figure 2. System and Horizontal Alignment Within an Education System

Adapted from Determining Alignment of Expectations and Assessments in Mathematics and Science Education by Webb (1997).

There are several traditional alignment methodologies, known typically as the Webb Alignment Method, the Surveys of Enacted Curriculum (SEC), and the Achieve Approach. All of these models examine specific criteria. These criteria include a combination of content focus (e.g., categorical concurrence, depth of knowledge consistency, range of knowledge correspondence, structure of knowledge comparability, balance of representation, and dispositional consonance), articulation across grades and ages, equity and fairness, and pedagogical implications. While each model provides the essential element of horizontal alignment of standards and assessments and can be applied to alternate assessments, each model also has unique features that can contribute to system alignment.

A new method to address the special nature of the AA-AAS is called Links for Academic Learning (LAL). Table 1 provides several references for all of these methods.

Table 1. Common Alignment Methodologies

Methodology	References
Achieve Approach (Achieve)	Achieve, Inc. (2001, April). Measuring up: A benchmarking study of the Minnesota Comprehensive Assessments. See www.achieve.org/dstore.nsf/Lookup/Minnesota-Benchmarking4-2001/$file/Minnesota-Benchmarking4-2001.pdf Rothman, R., Slattery, J. B., Vranek, J. L., & Resnick, L. B. (2002). Benchmarking and alignment of standards and testing. National Center for Research on Evaluation, Standards, and Student Testing. See www.cresst.org/
Links for Academic Learning (LAL)	Browder, D. M., Wakeman, S. Y., Flowers, C., Rickelman, R., Pugalee. D., & Karvonen, M. (2007). Creating access to the general curriculum with links to grade level content for students with significant cognitive disabilities: An explication of the concept. Journal of Special Education, 41, 2-16. Flowers, C., Karvonen, M., Browder, D., & Wakeman, S. (2007). Links for academic learning (LAL): A methodology for investigating alignment of alternate assessments based on alternate achievement standards. Available from cpflower@uncc.edu.
Surveys of Enacted Curriculum (SEC)	Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31, 3-14. Porter, A. C., & Smithson, J. L. (2002). Alignment of assessments, standards, and instruction using curriculum indicator data. See cep.terc.edu/dec/research/alignPaper.pdf Porter, A. C., Smithson, J., Blank, R., & Zeidner, T. (2007). Alignment as a teacher variable. Applied Measurement in Education, 20, 27-51.
Webb Alignment Method (Webb)	Webb, N. L. (1997a). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 6). Madison, WI: University of Wisconsin-Madison. See acstaff.wcer.wisc.edu/normw/WEBBMonograph6criteria.pdf Webb, N. L. (1997b). Determining alignment of expectations and assessments in Mathematics and Science Education: NISE Brief 1(2). See www.wcer.wisc.edu/archive/nise/Publications/Briefs/Vol_1_No_2 Webb, N. L. (1999). Alignment of science and mathematics standards and assessments in four states (Research Monograph No. 18). Madison, WI: University of Wisconsin-Madison.

Alignment Study Components

When extended standards are used by the state, these must be factored into the alignment study to confirm their link to the state’s content standards. Other criteria and components to consider are shown in Table 2. These components address some of the complications described earlier. For example, does the alignment study need to provide basic information on how many alternate assessment items are academic or within their intended domain (e.g., “really reading”)? Given that some prioritization of the standards occurs in developing alternate assessments, is the end result a system that reflects these priorities? Are they reflected in teaching as well as in instruction? Is there any differentiation across the grades or do students demonstrate the exact same tasks year after year in the alternate assessment?

The state is encouraged to review each component and determine whether it will be included in an alignment study and if so, how it will be evaluated. Whether the components are addressed by the available alignment models is shown in Table 3.

Table 2. A Summary of Alignment Criteria and Components for AA-AAS

Criteria/Components	Information Component Provides:	Alignment Areas Component Measures:
Content Focus
1. Academically Based	Is the content of the standard or item academic in nature? Can it be supported by content standards?	Extended Content Standards AA items
2. Reduction of Scope and Complexity	Does the reduction of content found in extended standards and AA match the intention of the state?	Test Blueprint or Assessment Frameworks to Extended Standards and Alternate Assessment
3. Content Concurrence	Are there an adequate number of extended standards and AA items/tasks within the different content standards/strands?	Extended Content Standard or Grade Level Content Standard (GLS) to Alternate Assessment item
4. Range of Knowledge	Are items aligned to multiple objectives that are nested within the content standards/strands?	Extended Content Standard or Grade Level Content Standard to AA items
5. Balance of Representation	Are the items evenly or emphasized in a way that matches the emphasis of the EXS/GLS?	Extended Content Standard or Grade Level Content Standard to AA items
6. Depth of Knowledge	EITHER what is the consistency between the cognitive demands of the standards and AA items OR what is the level of a knowledge taxonomy required by the student related to the performance and content of the standard/item?	Grade Level Content Standards Extended Content Standards AA items
7. Content Centrality	What is the quality or fidelity of the content of the GLS to the EXS or the EXS/GLS to the AA items?	Grade Level Content Standards to Extended Content Standards Extended Content Standards/Grade Level Content Standards to AA items
8. Performance Centrality	What is the fidelity of the cognitive demand or DOK of the GLS to the EXS or the EXS/GLS to the AA items?	Grade Level Content Standards to Extended Content Standards Extended Content Standards/Grade Level Content Standards to AA items
Articulation Across Grade Levels
9. Differentiation Across Grade Levels or Bands	Is there a change in emphasis of content across grade levels or bands (e.g., content of items do not repeat from 3rd to 8th grade)	Standards AA items
Equity and Fairness
10. Source of Challenge	Does the language of the item, item construction (e.g., any bias within an item), or accessibility of the item interfere with the student’s ability to answer correctly	AA items
11. Link of Achievement Standards to Grade Level Content Standards	To what extent are standards for achievement based on student performance?	Alternate Achievement Standards
Pedagogical Implications
12. Content of Instruction	What content is the focus of instruction for the population? What content do the teachers emphasize in their instruction?	Curriculum Indicators Survey* Surveys of Enacted Curriculum
13. Pedagogical Implications for All Areas (content, instruction, assessment)	What support are teachers receiving regarding aligning instruction and content standards? Regarding AA practices?	Quality Program Indicators Professional Development Curriculum Indicators Survey* Surveys of Enacted Curriculum

*Note: The Curriculum Indicators Survey follows a similar style and format to the Surveys of Enacted Curriculum but is written specifically for teachers of students who participate in an alternate assessment based on alternate achievement standards.

Table 3. Crosswalk of Components of Alignment and Alignment Methodologies

Criteria/Components	Achieve	LAL	SEC	Webb	Methodologies: Achieve Approach (Achieve) Links for Learning (LAL) Surveys of Enacted Curriculum (SEC) Webb Alignment Method (Webb)
Content Focus
Categorical concurrence		P		P
Content coverage (range and balance)	P	P	P	P
Depth of knowledge	P	P	P	P
Structure of knowledge	P	P		P
Content centrality (item)	P	P
Performance centrality (item)	P	P
Articulation Across Grade Levels
Cognitive soundness		P		P
Cumulative growth				P
Equity and Fairness
Source of challenge	P	P		P
Link of achievement standards		P
Pedagogical Implications
Instruction		P	P
Professional development		P	P
Instructional resources		P	P	P

Summary

This Policy Directions was written to describe the alignment issues surrounding AA-AAS and the components to consider when determining the degree of alignment for these assessments. Although alternate assessment requires that complex issues be considered when planning an alignment study, models do exist that states can use to provide both documentation for Peer Review and to glean information for future quality enhancement of the assessment system.

This information serves as an introduction to alignment considerations for states. A companion Policy Directions (Planning Alignment Studies For Alternate Assessments Based on Alternate Achievement Standards) delineates specific questions to ask when planning an alignment study for the AA-AAS. It also provides guidance to help maximize state resources.

Resources

Aligning Tests with States’ Content Standards: Methods and Issues. Bhola, D. S., Impara, J. C., & Buckendahl, C. W. (2003). Educational Measurement: Issues and Practices, 22, 21-29.

Alignment: Policy Goals, Policy Strategies, and Policy Outcomes. Baker, E. L., & Linn, R. L. (2000). Retrieved July 8, 2005, from http://www.cse.ucla.edu/products/newsletters/cresst_cl2000_1.pdf

Designing Content Targets for Alternate Assessments in Science: Reducing Depth, Breadth, and/or Complexity. Gong, B. (2007). Presentation at the web seminar Best Practice and Policy Consideration in Science Teaching and Testing for Students with Significant Cognitive Disabilities. Retrieved March 28, 2007, from http://www.nceo.info/Teleconferences/tele14

Horizontal and Vertical Alignment. Case, B. J., & Zucker, S. (2005). Retrieved June 5, 2005, from http://harcourtassessment.com/hai/Images/pdf/assessmentReports/HorizontalVerticalAlignment.pdf

State Standards and State Assessment Systems: A Guide to Alignment. La Marca, P. M., Redfield, D., & Winter, P. C., Bailey, A., & Despriet, L. H. (2000). Retrieved June 6, 2005, from http://www.ccsso.org/content/pdfs/ALFINAL.pdf

Top of page