| My Coursework Homepage |
Mark McDonough
Universally Designed Assessments
Abstract
Federal mandates have increasingly required states to assess all students, including those with disabilities. Many students require accommodations in order to access the standard administration of these large-scale assessments. Questions have arisen as to the fairness in deciding who receives accommodations and what type of accommodation they receive. Universal Design for Assessment provides a way to ensure that assessments are fair, valid, and accessible to all students from the start.
I. Introduction
In recent years, there has been nationwide, growing attention focused on large-scale assessments. The No Child Left Behind Act of 2001 mandated annual statewide testing of all students, including those with disabilities and limited English proficiency, in reading, math, and now science, in grades three through eight (NCLB website). The reauthorization of the Individuals with Disabilities Education Act of 2004 (IDEA) and the Assistive Technology Act of 2004 (ATA) both require “designing and delivering products and services that are usable by people with the widest possible range of functional capabilities” (Schafer and Liu, n.d., p. 2).
The standard administration of assessments is not appropriate for all students. Many students require accommodations which Tindal and Fuchs (1999, p.7) define as “changes in standardized assessment conditions.” With accommodations comes controversy. “Bielinski et al (2001) found that the use of accommodations may alter a test to an extent that the accommodated and un-accommodated items are no longer comparable” (Johnstone, 2003, p. 11). The challenge then becomes how to create a test that is both fair and valid for all users.
Enter Universal Design.
Universal Design is a concept developed by Ron Mace in the 1970s. The concept originally applied to architecture. Mace, a wheelchair user, believed that buildings should be designed and built so that all people regardless of ability, age, or situation could access them. It is usually more costly and less esthetically pleasing to retrofit a building to be more accessible. This concept of designing products and environments to be accessible to the widest range of people, including those with disabilities, has been applied to many fields. In education, it is usually referred to as Universal Design for Learning (UDL). “UDL provides a blueprint for creating flexible goals, methods, materials, and assessments that accommodate learner differences” (CAST Website). The purpose of this literature review is to examine current theory and practice of the principles of universal design as applied to assessments, especially large-scale assessments.
II. Discussion
As stated above, more states are testing more students, all students, including those with disabilities and limited English proficiency. The standard administration of assessments is not appropriate for all of these students. While a relatively small group of students require alternate assessments, there is a larger group that requires accommodations in order to take the standard assessment. “Repeated revisions in state accommodation policies is just one indicator of the controversy surrounding the need to provide accommodations for students to be able to participate and show their knowledge and skills in assessments” (Thompson and Thurlow, 2002, p.1).
The goal of universal design in assessment mirrors its goal in other fields: To make the product, in this case large-scale assessments, accessible from the start to the widest range of users while at the same time not compromising the assessments’ validity. An added benefit is a possible reduction in the number of necessary accommodations. “ Universally designed assessments are not intended to eliminate individualization, but they may reduce the need for accommodations and various alternative assessments by eliminating access barriers associated with the tests themselves” (Thompson, Johnstone, and Thurlow, 2002, p.3 ).
Empirical Studies
Relatively few studies on UDL and universally designed assessments have been carried out. This is likely due to the fact that these are relatively new concepts. UDL concepts and their implementation in the classroom, while catching on, are yet unknown to many educators.
Johnstone (2003, p.2) carried out a study “to determine whether the principles of universally designed assessment would more validity assess a sample of all students, and to specifically examine students who were economically poor, mostly English language learners, ethnic minorities, and students with disabilities.” The 231 sixth-grade participants each took two tests. One was traditionally designed; the other was a revision of the same test, adjusted to incorporate Universal Design principles. The results of the study showed that applying elements of Universal Design of assessment overwhelmingly increases student achievement on tests (Johnstone, 2003). Moreover, every subgroup, e.g. Navajo, Latino, Anglo, English language learners, “demonstrated improved performance on the Universally Designed test” (Johnstone, 2003, p.13). This project included a second, qualitative study that centered on students’ opinions of and experiences with the two tests. According to the students, the factors that are most important for high achievement on tests are “timing, readability, and recognizable materials” (Johnstone, 2003, p.13).
Large-Scale Assessment
The National Center on Educational Outcomes (NCEO) has been a leader in sponsoring research on UDL. Thompson et al (2002) from the NCEO performed a comprehensive review of literature on universal design, assessment and instructional design, and other related fields. They derived seven elements of universally designed assessments. While the team acknowledged that these elements would be refined over time, these elements have basically remained unchanged, and are referenced in much of the literature today.
Elements of Universally Designed Assessments
1. Inclusive assessment population 2. Precisely defined constructs 3. Accessible, non-biased items 4. Amenable to accommodations 5. Simple, clear, and intuitive instructions and procedures 6. Maximum readability and comprehensibility 7. Maximum legibility (Thompson, Johnstone, Thurlow, 2002, p. 11)
It is not within the scope of this review to further define and describe each of the above elements, however mention of “plan language” is appropriate since it does relate to several of these elements and to the spirit of universal design. Plain language, a concept used in assessment, is “language that is straightforward and concise” (Thompson, Johnstone, and Thurlow, 2002, p.4). Plain language strategies include reducing wordiness, using high frequency words, and avoiding multiple names for the same concept (Thompson, Johnstone, and Thurlow, 2002, p.4). As Thompson et al (2002, p.5) point out, “ It is critical … to use plain language when vocabulary level is not part of the construct being tested.”
Thompson was also the lead researcher in a follow-up report (Thompson, Johnstone, Anderson, and Miller, 2005). The second report took the elements of universally designed assessments of the first report and developed a list of specific considerations for the universal design of statewide assessments.
Considerations to Make Assessments More Universally Accessible 1. Incorporate elements of universal design in the early stages of test development . (Thompson, Johnstone, Anderson, Miller, 2005, p. 9)
Formative Assessment
This paper has, so far, dealt with large-scale assessments. There is another form of assessment, however, that is relevant to this discussion. A goal of a UDL is to embed assessment in the curriculum; in the process of instruction itself, by using flexible, usually digital, instructional media. This formative means of assessment provides accurate, ongoing information that informs and, ideally, improves instruction. It “functions less like a traditional test, and more like scaffolds with feedback” (Rose & Dole, 2001, p.5). An often cited example of this assessment involves CAST’s Thinking Reader.
(Hitchcock, Meyer, Rose, Jackson, 2002, p.4)
As more teachers implement curriculum based on the principles of UDL, it follows that there will be a wider understanding and implementation of both kinds of assessment: formative and summative.
(Thompson, Thurlow, Quenemoen, & Lehr, 2002, p.19)
The following study focuses on UDL interventions. 320 high school biology and algebra students participated. Some of their teachers used as many as six UDL interventions while others used none. In a closed-ended survey, “students reported strong levels of effectiveness, utility, and satisfaction related to the UDL interventions compared to their other academic classes. Both groups also consistently reported learning important and useful information, staying on-task, and working hard” (Kortering, McClannon, & Braziel, 2005, p.7). In an open-ended survey, students in both general education and special education reported very favorable views of UDL interventions: “90% of all participants expressed an interest in receiving more UDL interventions” (Kortering et al, 2005, p.8). The survey also found that the UDL interventions help students to learn and to use technology as an effective learning tool.
Further Considerations
There are themes closely related to UD for assessment that a more in-depth literature review might address. These themes include assessment accommodations for students with disabilities, computer-based testing, and online assessments. Because universally designed assessments can be administered by a computer, these themes are sometimes confounded.
SummaryThe concept of universally designed assessments is relatively new. Elements of, and considerations for, these assessments have been established, yet will likely be refined and updated as more research becomes available. The principles of universally designed assessments hold the potential not only to create more accessible, large-scale tests for all students, but also to provide a “more accurate picture of what all students know and can do so that educators can focus on the critical target of providing universally designed standards-based instruction” (Thompson and Thurlow, 2002, p. 5).
References
Bremer, Christine D., Clapper, Ann T., Hall, Tracey, Kachgal, Mera and Hitchcock, Chuck. (2002). Universal Design: A Strategy to Support Students’ Access to the General Education Curriculum. National Center on Secondary Education and Transition Research to Practice Brief. 1(3). Retrieved June 23, 2006 from http://www.ncset.org/publications/default.asp Dolan, R. P. and Hall, T. E. (2001). Universal Design for Learning: Implications for Large-Scale Assessment. IDA Perspectives 27(4). Retrieved June 25, 2006 from http://www.cast.org/system/galleries/download/byCAST/udlassessment.pdf. Hitchcock, Chuck. (2001). Balanced Instructional Support and Challenge in Universally Designed Learning Environments. Journal of Special Education and Technology. 16(4). Retrieved June 25, 2006 from http://jset.unlv.edu/16.4/hitchcock/first.html Hitchcock, Chuck, Meyer, Anne, Rose, David, and Jackson, Richard. (2002).Providing New Access to the General Curriculum Universal Design for Learning. Teaching Exceptional Children . 35(2) Retrieved June 26, 2006 from http://journals.sped.org/EC/Archive_Articles/VOL.35NO.2NOVDEC2002_TEC_Article%201.pdf. Johnstone, Christopher J. (2003). Improving Validity of Large-scale Test: Universal Design and Student Performance (Technical Report 37). National Center on Educational Outcomes. Retrieved June 23, 2006 from http://education.umn.edu/NCEO/OnlinePubs/Technical37.htm Kortering, Larry, McClannon, Terry and Braziel, Patricia. (2005). What Algebra and Biology Students Have to Say About Universal Design for Learning. National Center on Secondary Education and Transition Research to Practice Brief. 4(2) Retrieved June 23 from http://www.ncset.org/publications/default.asp Rose, David. (2001). Universal Design for Learning and Assessment. Journal of Special Education Technology 16(1). Retrieved June 25, 2006 from http://jset.unlv.edu/16.1/issuemenu.html Rose, David and Dolan, Bob. (2000). Universal Design for Learning. Journal of Special Education and Technology. 15(4). Retrieved June 25, 2006 from http://jset.unlv.edu/15.4/issuemenu.html Rose, David H. and Meyer, Anne. (2002). Teaching Every Student in the Digital Age: Universal Design for Learning. Center for Applied Special Technology Companion Website: http://www.cast.org/teachingeverystudent/ideas/tes/ Schafer, William D. and Liu, Min. (n.d.). Universal Design in Educational Assessments. Maryland Assessment Research Center for Educational Success. Retrieved June 24, 2006 fromhttp://marces.org/completed/Universal_DesignFNL.pdf. Thompson, Sandra J., Johnstone, Christopher J., Anderson, Michael E., and Miller, Nicole A. (2005). Considerations for the Development and Review of Universally Designed Assessments (Technical Report 42). National Center on Educational Outcomes. Retrieved June 23, 2006 from http://education.umn.edu/NCEO/OnlinePubs/Technical42.htm Thompson, Sandra J., Johnstone, Christopher J., and Thurlow, Martha L. (2002). Universal Design Applied to Large Scale Assessments (Synthesis Report 44). National Center on Educational Outcomes. Retrieved June 22, 2006 from http://education.umn.edu/NCEO/OnlinePubs/Synthesis44.html Thompson, S., and Thurlow, M. (2002). Universally Designed Assessments: Better Tests for Everyone! (Policy Directions 14). National Center on Educational Outcomes. Retrieved June 22, 2006 from http://education.umn.edu/NCEO/OnlinePubs/Policy14.htm Thompson, Sandra J., Thurlow, Martha L., Quenemoen, Rachel F., and Lehr, Camilla A. Access to Computer-Based Testing for Students with Disabilities. (2002). National Center on Educational Outcomes. Retrieved June 22, 2006 from http://education.umn.edu/NCEO/OnlinePubs/Synthesis45.html Tindal, G., and Fuchs, L.S. (1999). A Summary of Research on Test Changes: An Empirical Basis for Defining Accommodations. Lexington, KY: University of Kentucky, Mid-South Regional Center.
|