Transforming Education for Children with Disabilities Using Principles of Universal Toolkit Design for Assessment (UDA) to Design Accessible Learning Assessments ©2023 International Bank of Reconstruction and Development / The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org Some rights reserved. This work is a product of the staff of The Rights and Permissions World Bank with external contributions. The material in this work is subject to The findings, interpretations, and copyright. Because The World Bank conclusions expressed in this work do not encourages the dissemination of its necessarily reflect the views of The World knowledge, this work may be reproduced, in Bank, its Board of Directors or Executive whole or in part, for noncommercial purposes Directors, or the governments they as long as full attribution to this work is given. represent. Attribution—Please cite the work as follows: The World Bank does not guarantee the “World Bank. 2023. Toolkit: Using Principles accuracy, completeness, or currency of Universal Design for Assessment (UDA) of the data included in this work and to Design Accessible Learning Assessments. does not assume responsibility for any Washington, D.C: World Bank Group.” errors, omissions, or discrepancies in the information or liability with respect to the All queries on rights and licenses, including use of or failure to use the information, subsidiary rights, should be addressed to methods, processes, or conclusions World Bank Publications, The World Bank set forth. The boundaries, colors, Group, 1818 H Street NW, Washington, DC denominations, and other information 20433, USA; fax: 202-522-2625; e-mail: shown on any map in this work do not imply pubrights@worldbank.org. any judgment on the part of The World Bank concerning the legal status of any Photo credits (left to right): copyright, World territory or the endorsement or acceptance Bank. Bottom photo: Leonard Cheshire, UK. of such boundaries. Further permission required for reuse.  Contents Acronyms································································································································································································································ iv Acknowledgments··························································································································································································· vi Objectives ····························································································································································································································· 1 Key Messages·············································································································································································································· 2 1. Introduction····································································································································································································· 4 1.1. Tajikistan Pilot················································································································································································· 6 1.2. Purpose, Audience, and Structure························································································································· 8 2. Background··································································································································································································· 10 2.1. Categories of Assessment············································································································································· 11 2.2. Measuring Learning Outcomes for Learners with Disabilities············································12 3. Universal Design for Assessment ······································································································ 13 3.1. Understanding UDL·······························································································································································13 3.2. Understanding UDA·····························································································································································14 3.3. Intersections of UDL and UDA··································································································································16 3.4. Who Benefits from UDA···················································································································································· 17 3.5. Dispelling UDA Myths·························································································································································18 4. Principles in Delivering UDA····························································································································· 20 5. Understanding Accommodations and Modifications····················· 23 5.1. Understanding Accommodations······················································································································ 26 5.2. Understanding Modifications·································································································································· 29 Using Principles of UDA to Design i Toolkit Accessible Learning Assessments Contents 5.3. Accommodations versus Modifications·········································································································31 5.4. Ensuring Proficiency in Alternate Formats································································································ 34 5.5. Environmental Considerations································································································································ 34 6. Understanding Accessibility····························································································································· 36 6.1. General Accessibility Features································································································································ 36 6.2. Accessibility Features for Chalkboard-Administered Assessments······························41 6.3. Accessibility Features for Printed Text············································································································ 42 6.4. Accessibility Features for Technology-Based Assessments················································· 44 7. Universally Designed Classroom Assessments (Formative and Summative Assessments)·····································································46 7.1. Understanding Classroom Assessments·································································································· 46 7.2. Universally Designing Classroom Assessment Features························································ 48 8. Universally Designed EGRA and EGMA················································································· 50 8.1. Understanding Large-Scale Assessments·································································································51 8.2. Universally Designing EGRA and EGMA·······································································································51 8.3. Developing Literacy Assessments for Students Who Use Sign Language········· 53 8.4. Developing Literacy Assessments for Students who Use Braille··································· 55 8.5. Steps for Implementing UDA ·································································································································· 56 9. Using Assessment Data················································································································································ 61 10. Conclusion········································································································································································································ 68 References························································································································································································································ 69 Glossary··································································································································································································································· 75 Annex A: Images of Number Lines Used in Tajikistan UDA Pilot································································ 77 Annex B: Word Problem Images Used in Tajikistan UDA Pilot·······································································79 Annex C: Accessibility Checklist························································································································································ 80 Annex D: Additional Accommodations by Disability Type·················································································· 82 Using Principles of UDA to Design ii Toolkit Accessible Learning Assessments Contents List of Figures and Tables Figure 1: Assessment Accommodations 27 Figure 2: Accommodations versus Modifications Summary 34 Figure 3: Three Tiers for Instruction and Support 62 Table 1: Seven Elements of UDA 15 Table 2: Examples of How UDA Can Follow the Principles of UDL 16 Table 3: How UDA Supports All Students 17 Table 4: Accommodations and Modifications during Assessment Design, Implementation, and Scoring 25 Table 5: Accommodations versus Modifications 33 Table 6: Environment Considerations that Support UDA 35 Table 7: Possible Adaptations for EGMA Subtests 52 Table 8: Examples of Target and Access Skills 57 Table 9: Suggestions for Inclusive Practices by Tier 63 Table D.1: Suggested Accessibility Options for Blind or Low Vision Students 82 Table D.2: Suggested Accommodations for Deaf and Hard of Hearing Students 85 Table D.3: Suggested Accommodations for Deafblind Students and Students with Multiple Disabilities 87 Using Principles of UDA to Design iii Toolkit Accessible Learning Assessments Contents Acronyms AAC Augmentative and Alternative Communication ADHD Attention Deficit Hyperactivity Disorder CAST Center for Accessible Special Technologies CBM Curriculum-Based Measurement CCTV Closed-Captioned Television EGMA Early Grade Math Assessment EGRA Early Grade Reading Assessment EMIS Educational Management and Information System FCDO Foreign, Commonwealth, and Development Office FM Frequency Modulation HIC High-Income Country IDP Inclusive Development Partners IEI Inclusive Education Initiative IEP Individualized Education Plan IPEA Inclusive Primary Education Activity Using Principles of UDA to Design iv Toolkit Accessible Learning Assessments Acronyms LMIC Low- and Middle-Income Country LTA Learn Together Activity MoEYS Ministry of Education Youth and Sports MTSS Multi-Tiered System of Support NCEO National Center on Education Outcomes OPD Organization of Persons with Disabilities PASEC Programme d’Analyse des Systemes Educatifs de la CONFEMEN PISA Programme for International Student Assessment PISA-D Program for International Student Assessment for Development SEA-PLM Southeast Asia Primary Learning Matrix TaRL Teaching at the Right Level UDA Universal Design for Assessment UDL Universal Design for Learning USAID United States Agency for International Development WBG World Bank Group Using Principles of UDA to Design v Toolkit Accessible Learning Assessments Acronyms Acknowledgments This toolkit is a global knowledge product Stuart Cameron (Global Partnership for from the Inclusive Education Initiative (IEI)—a Education), and Asma Maladwala (UNICEF). multi-donor trust fund on disability-inclusive education managed by the World Bank, with The team also thanks external reviewers, support from the Norwegian Agency for collaborators, and contributors (in alpha- Development Cooperation (Norad) and the betical order) Angela Affran (Ministry of United Kingdom’s Foreign, Commonwealth, Education, Ghana), Dr. Deepa Srikantaiah and Development Office (FCDO). (USAID), Dr. Diane Rodriguez (Fordham University), Jennae Bulat (RTI International), The toolkit was conceptualized and Dr. Joy Banks (George Mason University), developed by the World Bank team, led by Dr. Leanne Ketterlin-Geller (Southern Charlotte McClain-Nhlapo and Ruchi Kulbir Methodist University), Dr. Linda Mason Singh. The team comprised (in alphabet- (George Mason University), Dr. Mary ical order) Anna-Maria Eftimiadis, Hanna Kealy (George Mason University), Dr. Katrina Alasuutari, Deepti Samant Raja, Morgan Chitiyo (University of North and Sophia Marie D’Angelo. In addition, Carolina Greensboro), Dr. Reshma Parvin the team is thankful for contributions from Nuri (Queen’s University), Sue Swenson Diego Luna Bazaldua. (Inclusion International), Dr. Socorro Herrera (Kansas State University), and Dr. The Inclusive Development Partners Yasmin Sitabkhan (RTI International). (IDP) partnered with the World Bank to conduct research and author the toolkit. The team is grateful for the support of The IDP team was led by Anne Hayes and Louise J. Cord (Global Director, Social comprised (in alphabetical order) Andrea Sustainability and Inclusion Global Practice) Shettle, Dr. Chris Johnstone (University of and Nikolas Myint (Practice Manager, Minnesota), Eileen Dombrowski, Kristin Social Sustainability and Inclusion. The Layton, and Dr. Valerie Karr. team is deeply grateful for their leadership and unconditional support in advancing This study would not have been possible inclusion of children with disabilities in without the support and guidance of Saodat education. Bazarova (World Bank, Tajikistan) in con- ducting the pilot in Tajikistan. Selvaraj Ranganathan and Susi Victor provided excellent editorial assistance. The team sincerely thanks our peer Alejandro Espinosa/sonideas.com assisted reviewers Laura Gregory (World Bank), with the design of the publication Using Principles of UDA to Design vi Toolkit Accessible Learning Assessments Acknowledgments Objectives • The purpose of this toolkit is to generate knowledge on how to develop and adapt assessment tools using principles of universal design that yield reliable and valid data and information to track the learning outcomes of marginalized learners, including learners with disabilities. • To provide background and evidence-based practices on developing inclusive and accessible assessments using the concepts of universal design for assessment (UDA) and uni- versal design for learning (UDL). • The toolkit is intended for World Bank staff, educational prac- titioners, and experts. • The toolkit is designed specifically considering the practical challenges faced by education systems in the Low-Middle Income Countries (LMICs). Using Principles of UDA to Design 1 Toolkit Accessible Learning Assessments Objectives Key Messages • The World Bank Group (WBG) has a corporate commitment of ensuring that all World Bank-financed education programs and projects are disability inclusive by 2025. • For children with disabilities, the learning crisis is twofold: equitable educa- tion access and quality of learning. • Children with disabilities are less likely to have foundational reading and numeracy skills than their peers with no functional difficulties. • Learning assessments are often not accessible to all learners because of the way assessments are developed. • Large-scale learning assessments are often created in a way that does not consider the cognitive, physical, and sociocultural needs of learners with disabilities and other marginalized groups. • Historically, learners with disabilities have been disadvantaged or excluded from various learning assessment practices. This lack of inclusion impedes their learning, as teachers and educational systems cannot adequately measure these students’ learning outcomes or progress. • Universally designed assessments are intended to allow the equitable participation of the widest possible range of students by considering the different learning needs of students and follow the general principle of ensuring that all students have the maximal opportunity to demonstrate how they are progressing on their educational goals. • In typical learning assessments, the assessment tasks, presentation of materials, or response requirements may be inaccessible to students with disabilities—or to any student. Teachers and policy makers who use UDA Using Principles of UDA to Design 2 Toolkit Accessible Learning Assessments Key Messages attempt to identify where these barriers may be present and create assess- ments that are both flexible and appropriately challenging for students. • For the development of this toolkit, the accessible Early Grade Math Assessment (EGMA) pilot was conducted in Tajikistan. Lessons from the pilot have informed the development of the toolkit. This is an example of just one type of assessment, but the toolkit provides suggestions for other assessments as well. • Universally designed assessments: | Ensure that assessments are designed in a way that includes all students regardless of disability status, language, gender, and so on. | Focus on clearly designed test items and remove irrelevant cognitive, sensory, emotional, and physical barriers. | Provide built-in accessibility in test items. Assessments are reviewed to remove items that may be biased against students with disabilities and other marginalized groups. | Allow accommodations (for example, braille and sign language) without changing the test construct. | Ensure instructions are simple, clear, and presented in understandable language. | Allow maximum readability and comprehension by following plain language guidelines to produce readable and comprehensible text (for example, limited sentence length and avoiding unnecessary or difficult words). | Support maximum legibility by providing information (including text, tables, figures, illustrations) in an ‘easy to decipher’ format (for example, color contrast and font size). • This toolkit provides a detailed ‘how to’ guidance on designing and piloting UDA tools. Using Principles of UDA to Design 3 Toolkit Accessible Learning Assessments Key Messages Photo credit: World Bank 1. Introduction Assessment is a critical part of all education systems. The World Bank Framework on Assessment defines the term as “the process of gathering and evaluating information on what students know, understand, and can do in order to make an informed decision about the next steps in the educational process” (Clarke 2012, 1). But assessment goes beyond simply testing a student’s knowledge; it helps educators understand students’ needs, set standards, and evaluate progress. Therefore, assessment should be designed to not focus on students’ failures but instead provide opportunities for students to demonstrate knowledge. Assessments also help provide national data on how the education system works and what changes must be incorporated. Done well, assessment can motivate and direct teachers’ classroom instruction by letting them know what areas of instruction work and where they may need to place more focus. Essentially, when assessment is done correctly, it can help improve the learning experience for all students. Using Principles of UDA to Design 4 Toolkit Accessible Learning Assessments Introduction However, although more teachers are being may encounter long-term implications if trained on how to educate all students, they do not participate in assessments or if comparatively less training and guidance they do poorly due to inadequate support. have been provided on how to assess the Additionally, these implications may affect learning of students with disabilities. As their college entrance, skills development, countries work to expand access to edu- and workforce opportunities. cation for all learners, more programs are beginning to implement universal design for Learning assessments are often not learning (UDL).1 UDL is an evidence-based accessible to all learners because of approach that supports learning outcomes the way these assessments are devel- for students with and without disabilities oped. Learning assessments, especially and is based on the premise that there large-scale assessments, were originally is great variability in how children learn designed to efficiently sort students. In (Meyer, Rose, and Gordan 2014). However, these assessments, standard presenta- learners with disabilities have historically tion and response formats were deemed been disadvantaged or excluded from essential for perceived “fairness” (Linn various learning assessment practices. This 2001). As a result, assessments are often lack of inclusion impedes their learning, as created in a way that does not consider teachers and educational systems are not the cognitive, physical, and sociocultural able to adequately measure these students’ needs of learners with disabilities and learning outcomes or progress. Therefore, other marginalized groups. This often even in schools with accessible learning leads to the development of inaccessible environments, assessments are often not tools and assessment practices that do accessible and, thus, do not adequately not appropriately gauge the educational capture the learning of students with dis- progress of learners in the classroom. abilities. If assessments do not accurately However, if assessments are designed capture the learning of all students, includ- with the diverse needs of students in mind, ing those traditionally marginalized within more accurate information can be gathered the education system, diverse populations on learning within the classroom—for all within a country, region, or classroom students. Furthermore, though a universally are at risk of being left behind. Being left designed assessment will not eliminate out or marginalized in the assessment the need to provide accommodations to process can have detrimental impacts on students with disabilities, when universal students with disabilities. These students design for assessment (UDA) is used in 1 The “UDL approach recognizes that each student learns in a unique manner. UDL consists of a set of principles providing teachers and other staff with a structure for creating adaptable learning environments and developing instruction to meet diverse needs of all learners. It involves creating engaging classroom environments; maintaining high expectations from all students while allowing multiple ways to meet expectation flexible ways of learning flexible curriculum; empowering teachers to think differently about their own teaching; and focusing on educational outcome for all, including students with disabilities” (World Bank, Inclusion International, and Leonard Cheshire 2019). Using Principles of UDA to Design 5 Toolkit Accessible Learning Assessments Introduction the design of an assessment, the need for Bank team worked closely with IDP accommodations is often reduced. and on-the-ground partners such as Chemonics International and EdIntersect This toolkit attempts to fill the information (supported by USAID) to organize the and knowledge gap around the applica- accessible EGMA pilot in tandem with LTA. tion of UDL principles to design inclusive The accessible EGMA pilot was highly learning assessments. The toolkit presents consultative and included several meetings an evidence-based approach for the devel- with key stakeholders and organizations opment of accessible assessment practices of persons with disabilities (OPDs) in that can be used by education practitioners, Tajikistan, including convening the original assessment experts, and education spe- EGMA development group, and meeting cialists. The toolkit development process with the Tajikistan National Working Group included applying key principles to design on EGMA to discuss the purpose of the accessible Early Grade Math Assessment pilot, UDA, and review of the accessible (EGMA) and piloting the assessment in subtests. As part of this project, enumer- mainstream schools in Tajikistan. Key ators were trained in Russian and Tajik on lessons from the pilot have been integrated the accessible version of EGMA. Although into the toolkit. several options were created to help make EGMA more accessible, the Tajikistan working group3 decided to focus on two 1.1. Tajikistan Pilot EGMA subtests: Number Discrimination and Word Problems. The accessible version To anchor the UDA toolkit in a country con- of the assessment featured the following text, the Inclusive Education Initiative (IEI), changes: with support from Inclusive Development Partners (IDP), identified Tajikistan to pilot • For the Grade 2 and Grade 4 Number the accessible version of EGMA. 2 Selecting Discrimination subtest, students Tajikistan for the accessible EGMA pilot received a number line and an addi- was both opportune and strategic as devel- tional practice opportunity.4 Students opment partners such as the United States could also use counting sticks, an Agency for International Development abacus, or paper and pencil to solve (USAID) were piloting a standard EGMA problems. This allowed students to baseline activity through a USAID-funded select the way in which they wanted to Learn Together Activity (LTA). The World present their knowledge by providing 2 EGMA was selected as LTA was developing and piloting EGMA tools to be used for a baseline, midline, and endline assessment for the project. This timing allowed for the development of accessible EGMA during the initial stages of implementation. 3 The working group comprised several OPDs representing a variety of disability types, including OPDs for the blind and deaf. 4 Number lines were selected instead of other forms of representation as number lines were currently used in the classroom in Tajikistan. Using Principles of UDA to Design 6 Toolkit Accessible Learning Assessments Introduction a variety of options. Annex A provides • The pilot showed that Tajik language images and image descriptions of the learners benefited the most from number lines used. the accessible EGMA and that Tajik students were reported to be excited • For the Grade 2 and Grade 4 Word to be provided with choices during Problems subtest, students were the administration of the accessible allowed to choose the context of the EGMA.6 story (sports, food, or a school bus). Word problems included images for • Girls in Grade 4 performed significantly Grade 2, and students could also use better on the accessible EGMA over the counting sticks, an abacus, or number standard EGMA. lines to solve problems. Annex B pres- ents some of the images used in Grade • Lowest quintile scores showed a slight 2 Word Problems. advantage for Grade 4 test takers using the accessible EGMA. No differences The Number Discrimination and Word were found for students in Grade 2. Problem subtests were selected because they could be adapted without changing • An accessible EGMA only required an constructs while still allowing opportunities additional 1 minute to administer and, for additional accessibility. Findings from thus, did not require significant addi- this pilot revealed the following:5 tional time. • Students in Grade 4 achieved higher • Using some of the accessibility features scores using the accessible EGMA to receive and express information was compared to the standard EGMA new for students, and students may on both subtests. Students in Grade have benefited from completing an 2 received almost identical scores accessible EGMA coupled with addi- on both forms of the assessment. tional UDL support in the classroom. However, the results of the full pilot revealed that the subtest on which the • Although the concept of UDA was accessible EGMA was based may have received well by all stakeholders, suffi- been too easy for Grade 2 students cient time is needed to introduce new and, thus, needed to be revised in tools and testing administration pro- general. cesses to ensure that all stakeholders 5 These findings are related to EGMA and the Tajikistan pilot which addresses UDA and being more inclusive for all learners. Annex D provides recommendation for learners who are blind, deaf, or deafblind for what types of accommoda- tions may be needed for these learners. 6 School in Tajikistan recently shifted from Uzbek language to Tajik language as a language on instruction which may have lowered student reading levels initially. Evidence from USAID programs show that Tajik language learners have been seen as having lower performance on literacy scores in the past. Using Principles of UDA to Design 7 Toolkit Accessible Learning Assessments Introduction have buy-in and support the revised the standard EGMA. The Government of UDA. Tajikistan is also committed to inclusive education and showed interest and support • The pilot shows the importance of of the pilot. UDA as it benefited girls and Tajik language learners and was the pre- Future administrations, however, would ferred assessment of the participating need to consider target populations (in this students. Additional pilots and studies case, students with disabilities) and plan for will be needed in other countries as piloting where students are receiving their well to contextualize EGMA and show education. In this case, future EGMA pilots the importance of having accessible should take place in special schools7 to EGMAs for all students. ensure that the population of students with disabilities is represented in assessment It is important to note that the UDA pilot development, implementation, and data was administered in general education analysis/interpretation. settings in Tajikistan as access to segre- gated schools was not feasible. Because Additional lessons learned from the pilot many students with identified disabilities were integrated into this toolkit. The toolkit are currently educated in segregated provides suggestions not only on how to settings in the country and identification make large-scale assessments such as practices in the general education settings EGMA accessible but other formative and are nascent, the pilot was limited as it could summative assessments as well. not accurately capture how learners with identified disabilities benefit from UDA in the classroom. However, it can be assumed 1.2. Purpose, Audience, and that learners with unidentified disabilities, Structure albeit potentially high-incidence disabil- ities (challenges with vision and hearing, The purpose of this toolkit is to provide learning disabilities, attentional disabilities, background and evidence-based prac- emotional disabilities, speech disabilities, tices on how best to develop inclusive and so on), were in the classroom and ben- and accessible assessments using UDA efited from the accessible EGMA. Tajikistan and UDL concepts. The audience of this was selected for the pilot as it was devel- toolkit is World Bank staff, educational oping EGMA, so it was an opportune time practitioners, and experts. The toolkit may to roll out an accessible EGMA and assess also help guide donors, governments, and the differences in scores compared to disability advocates on how they can adapt 7 IEI-World Bank promotes the inclusion of children with disabilities in mainstream education; however, IEI also recog- nizes that segregated schools exist in national education systems. Based on the country context, these schools should be included in pilots to improve the validity and utility of any new accessible learning assessment. Using Principles of UDA to Design 8 Toolkit Accessible Learning Assessments Introduction their own work to better gauge the learning and skills in schools. Section 6 discusses of all students in the classroom. accommodations and modifications and the differences between the two, while Section The structure of this toolkit goes from 7 discusses different accessibility features, broad to narrow. The beginning of Section including those for specific disability types, 2 provides an overview of assessment and such as learners who are deaf or blind. In its purposes and introduces some common Section 8, case examples describe how to challenges related to the measurement align the Early Grade Reading Assessment of learning outcomes for students with (EGRA) and EGMA with UDA principles. disabilities. Section 3 then introduces the Section 9 provides teachers examples of concept of UDA, and section 4 provides how to incorporate UDA principles into their specifics on the principles of UDA that can classroom assessments, which are often be applied across educational settings more flexible and formative than large-scale worldwide, promoting accessibility for assessments. The toolkit concludes with students. Section 5 provides an overview Section 10, which discusses how assess- on why the accessibility of assessments is ment data can be used. Assessment data part of a larger inclusive education agenda can answer many questions about students’ aimed at providing equal opportunities for learning; however, every assessment is also all students to demonstrate their knowledge limited in the information it can provide. Using Principles of UDA to Design 9 Toolkit Accessible Learning Assessments Introduction Photo credit: World Bank 2. Background This section reviews... • The different types of assessments that can be implemented in the classroom • The current state of assessment for students with disabilities Assessment is any practice that helps teachers or systems understand the knowledge or skills that students have developed (Newton 2007). Students can be assessed in many ways such as asking them to show their work, having a conversation with them, and testing them. UDA is an educational strategy aimed at reducing barriers for students to demonstrate their knowledge and skills (Hanna 2005). The general principle behind UDA is that all students should have the maximal opportunity to demonstrate how they are progressing on their educational goals. The assessment tasks, presentation of materials, or response require- ments may be inaccessible to students with disabilities—or to any student. Teachers and policy makers who embrace UDA attempt to identify where these barriers may be present and create assessments that are both flexible and appropriately challenging for students. Using Principles of UDA to Design 10 Toolkit Accessible Learning Assessments Background 2.1. Categories of after an individual lesson or after an entire Assessment school year to measure student progress toward learning goals (RTI International Several types of assessments can be imple- 2022). Common examples of summative mented within an education system, and assessments are quarterly or annual they vary depending on the goal for which examinations. A danger of relying only on they are implemented. summative assessments for information is that it is sometimes difficult to ensure Formative assessments. Teachers that assessments closely align with the use formative assessments to inform content if students are measured through ongoing day-to-day instruction to help a single summative test. As a result, the achieve intended educational outcomes development of some skills (for example, a (AERA, APA, and NCME 2014). Formative student’s ability to create oral persuasive assessments help educators check stu- arguments) can be underemphasized in dents’ understanding and learning (RTI instruction to ensure that standards can International 2022) and identify gaps be measured by an end-of-year summative in student knowledge that need to be test (AERA, APA, and NCME 2014). addressed in daily classroom instruction. Although it is primarily used as an assess- Large-scale assessments. These ment delivered by teachers to students, assessments provide information on formative assessment can also be used overall performance levels and trends by students as a form of self-assessment in the education system as an aid to to help foster their motivation (Andrade policy and decision-making (Clarke and Cizek 2010). Since the main goal of and Luna-Bazaldua 2021). Two types of formative assessments is to inform teacher large-scale assessments currently used in instructional strategy or student self-mo- many low- and middle-income countries tivation, it is generally recommended that (LMICs) are EGRA and EGMA, to assess formative assessments not be graded learners on reading and math within (Andrade and Cizek 2010). USAID-funded early-grade projects. Other examples of regional and international Summative assessments. Such assess- large-scale assessment include Program ments are used to assess a student’s for International Student Assessment for knowledge and skill acquisition after the Development (PISA-D), Southeast Asia teacher completes instruction (AERA, APA, Primary Learning Matrix (SEA-PLM), and NCME 2014). These assessments can and Programme d’Analyse des Systemes be administered at various times, including Educatifs de la CONFEMEN (PASEC). Using Principles of UDA to Design 11 Toolkit Accessible Learning Assessments Background 2.2. Measuring Learning Additionally, students with identified Outcomes for Learners disabilities worldwide are often excluded from testing altogether. This is especially with Disabilities true for tests that may reflect badly on a teacher, school, or program; in these cases, Although classroom instruction is educators are often incentivized to exclude becoming more inclusive, ensuring that students who may score poorly, such as classroom assessment measurements students with disabilities (Erickson and are inclusive of all students with disabil- Thurlow 1996). For example, students with ities continues to be a challenge. This is multiple disabilities and intellectual dis- often due to the lack of identification of abilities are excluded from the Program for students with disabilities (that is, teachers International Student Assessment (PISA) are not aware that certain students have test on the assumption that students with disabilities and could benefit from testing intellectual and multiple disabilities access accommodations), the inconsistent use of only segregated schools, which are further testing accommodations for those who are excluded from the overall sample (World identified as having disabilities, and the Bank, Inclusion International, and Leonard lack of time and resources on the part of Cheshire 2019). Students with disabili- teachers (Buzick and Laitusis 2010). These ties may instead be offered a modified challenges create barriers in comparing assessment, which may include testing the performance measurements of students student using content from an earlier grade with disabilities to those of their peers or deleting some items from the test (AERA, without disabilities. Furthermore, these APA, and NCME 2014). In these cases, challenges can limit the usability of data although students with disabilities are still over a long time and continue to exclude participating in testing, their results have students with disabilities within the data lost comparability, as their results cannot (Buzick and Laitusis 2010). be compared to their peers. Using Principles of UDA to Design 12 Toolkit Accessible Learning Assessments Background Photo credit: World Bank 3. Universal Design for Assessment This section reviews... • The principles of UDL • How UDL and UDA intersect • The key elements of UDA • Who can benefit from UDA • Common myths about UDA 3.1. Understanding UDL UDL is based on the premise that there is tremendous variability in how students learn (Meyer, Rose, and Gordon 2014). Based on both neuroscience and learning sciences, UDL uses three basic principles: Using Principles of UDA to Design 13 Toolkit Accessible Learning Assessments Universal Design for Assessment 1. Multiple means of engagement. All students are motivated to learn in different ways. Providing multiple options and choices through story reading, exercises, or group practice is one of the best ways to motivate students. Increasing student moti- vation helps increase student focus. 2. Multiple means of representation. Students learn differently, which means that teachers must present information in multiple ways. Some students learn best by hearing, seeing, writing, or acting out information. Therefore, instruction should offer a variety of ways to learn new information and match the students’ strengths. 3. Multiple means of action and expression. Because students learn differently, teachers should try offering a variety of options and allowing students to select the way they prefer to show knowledge. This can include using oral responses, written responses, technology, sign language, and other ways to demonstrate what a student has learned (CAST 2018). UDL in Practice Use of UDL strategies features in various LMICs as well as high-income countries (HICs). UDL promotes motivating students, presenting information, and allowing students to express their understanding in various ways. UDL is specifically referenced as a foundational inclusion approach in Ghana’s Inclusive Education Policy and in the Rwanda Basic Education Board’s Strategic Plan 2020–2025. In Liberia, starting in 2022 with support from the USAID TESTS Activity, at least 3,500 preservice teachers at eight colleges and universities will complete a 3-credit course on Introduction to UDL. In Cambodia, the Inclusive Primary Education Activity (IPEA) is working with the Cambodian Ministry of Education Youth and Sports (MoEYS) to embed UDL strategies in early grade literacy instruction and teacher training activities. Collectively, these examples demonstrate that momentum is building among various nations to support instructional strategies that benefit learners with and without disabilities. 3.2. Understanding UDA Universally designed assessments are intended to allow the equitable participation of the widest possible range of students (Thompson and Thurlow 2002). Such assessments that do not change the construct can produce higher levels of performance in low-performing students including those with disabilities and students who take tests in languages other Using Principles of UDA to Design 14 Toolkit Accessible Learning Assessments Universal Design for Assessment than their own by more accurately demonstrating their knowledge and skills (Downing 2005; Haladyna and Downing 2004; Johnstone 2003). This is because universally designed assessments are developed to consider the different learning needs of students. There are seven elements of UDA, as described in Table 1. What Is Assessment Construct? A construct is the concept or content being tested. When making tests accessible, it is important to keep the construct the same but determine different ways to make the construct more accessible. Keeping constructs the same allows for comparative data between students. TABLE 1: Seven Elements of UDA Element Description Inclusive assessment Tests are designed in a way that they can include all students population regardless of disability status, language, gender, and so on. Precisely defined constructs Test constructs are clearly designed and remove irrelevant cognitive, sensory, emotional, and physical barriers. Accessible, non-biased items Accessibility items are built into the test, and assessments are reviewed to remove items that may be biased against students with disabilities and other marginalized groups. Amenable to accommodations Test allows accommodations (for example, braille and sign language) without changing the construct. Simple, clear, and intuitive All instructions and procedures are simple, clear, and presented in instructions and procedures understandable language. Maximum readability and Plain language guidelines are followed to produce readable and comprehensibility comprehensible text (for example, limited sentence length and avoiding unnecessary or difficult words). Maximum legibility Information (including text, tables, figures, illustrations) is easy to decipher and interpret (for example, color contrast and font size). Source: Thompson and Thurlow 2002. Using Principles of UDA to Design 15 Toolkit Accessible Learning Assessments Universal Design for Assessment 3.3. Intersections of UDL and UDA Whether assessment is on an international scale or informally performed in a classroom, it should always be linked to learning and guided by accessibility. This toolkit has specific examples for how to make assessments more accessible, and in many ways, these are guided by the abovementioned three principles of UDL (that is, multiple means of engage- ment, multiple means of representation, and multiple means of action and expression). Flexibility in how assessments provide students information and how students can respond are two cornerstones to UDA and are informed by UDL directly. UDL and UDA may have slightly different applications (for example, a class learning activity may differ from how a teacher decides to assess the outcomes of the activity), but both are informed by a commit- ment to give all students the opportunity to both learn and show what they have learned. Opportunity, in this case, comes from ensuring that classroom activities (UDL) and the ways in which those activities are assessed (UDA) promote accessibility. This toolkit provides strategies to develop universally designed assessments that integrate the three principles of UDL and UDA. Table 2 provides an overview of how UDA could support the principles of UDL. The approach aims to promote accurate assessments of the widest possible range of students. TABLE 2: Examples of How UDA Can Follow the Principles of UDL UDL Principle Examples of UDA Multiple means of engagement Students are provided with choice of themes. For example, in Tajikistan UDA EGMA pilot word problem items, students could choose themes for word problems (sports, food, animals, or transportation). Subsequent word problems were then focused on the topic that most interested the student. Multiple means of representation Students are presented information in different formats such as verbal and written instructions and provided images to reinforce concepts. For example, in the Tajikistan UDA EGMA pilot, students were read the instructions, provided with written instructions, and presented with images to reinforce topics. Multiple means of action and Students can demonstrate learning in different ways including expression pointing, using verbal response, and using manipulative and written response. Students could answer by demonstrating with manipulatives, responding orally, or writing answers to items on the Tajikistan EGMA pilot. Enumerators recorded all answers from students. Using Principles of UDA to Design 16 Toolkit Accessible Learning Assessments Universal Design for Assessment 3.4. Who Benefits from UDA Similar to UDL, UDA was developed to support the learning of students with disabilities as well as learning assessments. However, research shows that these approaches may also help other students, including students speaking languages other than that spoken in schools (Abedi 2021). Further research is needed for other groups and UDA, but early evidence suggests that UDL has been effectively utilized for students who have experienced trauma (Salvador and Culp 2022), students experiencing poverty (Katz 2013), and potentially other marginalized groups, such as overaged students. Table 3 provides an overview of how UDA might support the learning of a variety of students who are often marginalized within education. As implementation of UDA principles emerges, so will a global evidence base on UDA’s potential and limitations. TABLE 3: How UDA Supports All Students Marginalized Group Benefits of UDA Students with disabilities With the principles of UDL embedded in assessments, learners with disabilities can receive and express information in different ways. For example, a student with limited writing or challenges with writing can point to the response or use other accommodations to show their knowledge. Girls and boys Providing choice of topics that students find most interesting can reduce the likelihood of having assessments that are not gender responsive. Ethnic and linguistic minorities Students who are learning the language of instruction may have challenges only receiving written or oral instructions. By providing and allowing both, minority language learners can benefit from UDA by being able to express their knowledge verbally or in writing. Having choice of topics relevant to different cultures can also increase motivation. Students who have experienced Students who have experienced trauma may have difficulty focusing trauma and do best when they are able to control various aspects of their lives. With multiple means of representation, minority language learners can benefit from UDA as they can express their knowledge verbally or in writing. This may also allow them to avoid activities that may trigger traumatic experiences. Using Principles of UDA to Design 17 Toolkit Accessible Learning Assessments Universal Design for Assessment Marginalized Group Benefits of UDA Students experiencing poverty Students who are hungry may need help with focusing and have a harder time attending to tasks over long periods. By allowing flexibility in the timing of assessments, increasing motivation through choice of topic, and allowing frequent breaks, these students may demonstrate their understanding of the assessment content more accurately. It is also important to avoid questions that may introduce trauma or anxiety (such as money-related exercises) or may be biased because of differential life experience (Fleisher 2022) Overaged students Overaged students may be in lower grade levels of instruction, but their interests are often similar to their same-aged peers. In countries with a large number of overaged students, UDA can provide different topics and illustrations that align with the interests of overaged student’s, which may reduce the embarrassment often associated with being overage. 3.5. Dispelling UDA Myths Several harmful myths about universally designed assessments for learners with disabilities are described below. • Myth 1: It will take too much time to administer a UDA test versus a traditional test. A recent pilot study of EGMA in Tajikistan tested this myth, and the difference in the time it took to deliver the UDA EGMA was not statistically significant. In this study, a ‘standard version’ of EGMA averaged 26–27 minutes for administration, depending on the language of administration. The UDA version of the assessment averaged 27–30 minutes, depending on language of administration as well. • Myth 2: UDA signifies easier test questions for students. A core principle of UDA is that tested constructs remain the same in accessible versions of assessments. Therefore, UDA tests are not easier, do not give answers away to students, and do not give hints to students. Rather, UDA tests aim to maintain the same level of rigor as the tests currently used but eliminate barriers to students participating in and demon- strating their knowledge on assessments. Using UDA may produce higher levels of performance for all students as UDA more accurately reflects the students’ knowledge and skills. The pilot showed similar results where test score improved for most students, particularly for girls and Tajik language learners. Using Principles of UDA to Design 18 Toolkit Accessible Learning Assessments Universal Design for Assessment • Myth 3: Only students with disabilities require UDA. UDA is an approach to make assessments more accessible for all students but especially focuses on removing barri- ers for students with disabilities. • Myth 4: Revising assessments to make them more accessible is too complicated. The abovementioned pilot in Tajikistan demonstrated that UDA was an easy concept to grasp for stakeholders from a wide range of backgrounds. Teachers, measurement professionals, and policy makers all found the general concepts of UDA to be intuitive. Rather than thinking UDA was too complicated, stakeholders in Tajikistan offered lively debate over the best ways, among multiple options, to make assessments more accessible. • Myth 5: UDA is too expensive. Although there are associated costs with reviewing the tests to ensure they are designed inclusively, the up-front costs of designing assess- ments for accessibility may be far less than retrofitting assessments once they have already been created. Using Principles of UDA to Design 19 Toolkit Accessible Learning Assessments Universal Design for Assessment Photo credit: World Bank 4. Principles in Delivering UDA This section reviews... • Core principles that should be followed for all students when administering UDA. When developing assessments for any student, it is important to follow five basic princi- ples of UDA, as described in the following paragraphs. Principle 1: Presume competence. This is a core element of inclusive education as it rein- forces the key belief that all students can learn and should be an integral part of the general education system. Following this principle in assessments avoids misperceiving that stu- dents cannot participate in assessments as they do not have the capacity to learn and, thus, Using Principles of UDA to Design 20 Toolkit Accessible Learning Assessments Principles in Delivering UDA cannot demonstrate their learning. Rather, areas; and ethnolinguistic and other minori- participation in assessment systems may ties). In addition, an assessment should provide data that better identify gaps in pol- strive to promote gender equity and diversity icy, instruction, and opportunity. Excluding within the content by having gender parity in students from any aspect of education on images and characters as well as including the basis of disability label (or any other persons with disabilities and other minori- demographic consideration) is problematic tized demographic groups in empowering because demographic descriptors are ways. Recognizing and promoting diversity socially constructed and do not predict requires two commitments from educational a student’s capacity to learn or demon- systems. The first commitment is to include strate knowledge. Rather, as assessment all students in the assessment population. designers construct learning assessments, Excluding students from any educational presuming competence means considering activity (including assessment) belies their the entire population of students for partic- educational rights. The second commitment ipation. In other words, the goal is to have is to collect demographic data to ensure as many students take the same tests that all demographic groups have access to as possible to allow for comparable data assessment. Demographic data collection and tools to measure students’ individ- allows for comparative analyses that may ual progress. A ‘presume competence’ highlight inequities in opportunity. Specific approach to learning assessment assumes demographic categories may differ slightly that all students can participate in national by country but can be informed by the education systems if barriers to those demographic data that are often collected in systems are removed. educational management and information systems (EMISs). Principle 2: Recognize and promote diver- sity. When developing national, formative, Principle 3: Ensure confidentiality. It is vital and summative assessment systems, all stu- to maintain student confidentiality through- dents within the country must be included. out the assessment process. This includes For example, if assessments do not include maintaining the confidentiality of student learners with isabilities or other demographic records, including assessment results and groups, the data collected do not provide accommodation needs. Confidentiality is a holistic or accurate picture of learning also an important aspect of administering outcomes. Therefore, all students should be national as well as international tests, such included in assessments, including those as EGRAs and EGMAs. Therefore, if a with different types of disabilities (students student needs to give oral responses on an who are deaf or hard of hearing, are blind or assessment, the student should not com- have low vision, are deaf and blind, or have plete the assessment in the presence of their an intellectual disability;8 students from rural peers (RTI International 2016). 8 These are groups of individuals with disabilities who have traditionally been marginalized within the education system. Using Principles of UDA to Design 21 Toolkit Accessible Learning Assessments Principles in Delivering UDA Principle 4: Do no harm. Assessment data NCME (2014) note that for assessments should be used to have a better picture of to be effective they must be valid, reliable, national and individual learning outcomes and fair. Further, large-scale assessments with the ultimate goal of improving instruc- must have transparent specifications and, tion and learning for all students. Although while testing content, align with national assessments can be a useful way to identify standards. All scoring and scales should students who are struggling with particular reflect the intended purposes of the assess- content, information from assessments ment (that is, if an assessment is focused should never be used as a sole tool to on specific criteria, establishing normative diagnose a student as having a disability judgments may be inappropriate). To this or as a reason to remove a student from end, all test users and implementers must the classroom environment or place a be trained on the usage and purposes of student in an alternative setting. Information assessments before implementation. Finally, obtained through assessments should be processes for documenting assessment used to support the learning of students results and rules for interpreting scores within inclusive settings and should not for educational decision-making must be used as data to promote segregation. be established. AERA, APA, and NCME (2014) also indicate that test takers have Principle 5: UDA aligns with best rights in the assessment process. These practices in testing. UDA represents rights include the right to preparation for a new opportunity for creating acces- the assessment and the right to privacy of sible assessments for all students, but assessments results. Test taker rights are implementing UDA principles does not upheld by responsibilities of test users, preclude assessment designers from fol- which include knowing the validity of lowing long-accepted assessment design assessments used and upholding privacy of principles. For example, AERA, APA, and results. Using Principles of UDA to Design 22 Toolkit Accessible Learning Assessments Principles in Delivering UDA Photo credit: World Bank 5. Understanding Accommodations and Modifications This section reviews... • Accommodations that can be used to support students when administering assessments • What a modification is and when it should be used for learners with disabilities • The differences and similarities between accommodations and modifications. Accommodations and modifications are two similar but distinct supports that are often used by learners with disabilities. Both accommodations and modifications are considered to be adaptions that are designed to reduce barriers to the original tests (AERA, APA, and NCME 2014) with modifications changing the construct of what is being tested while Using Principles of UDA to Design 23 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications accommodations provide support but do to demonstrate knowledge and thus do not not change what is being tested. Shifting provide unfair academic advantages for the construct of the test should be used learners with disabilities. sparingly if at all. These two adaptions are described more below. Modifications are changes to what a student is assessed on. This typically Accommodations are changes in how a arises when instructional modifications student accesses information and demon- are made to what a student is expected strates learning. The World Bank defines to learn. Instructional modifications may accommodations as considerations for allow students to participate in the regular accessibity or specific supports that are education classroom and demonstrate their different from person to person, dpending learning even if it is at a different level than on the type of disability and other factors their peers without disabilities. Modified (World Bank 2021). Accommodations do content should be age appropriate and not change the content, constructs, or similar to the content of their peers and instructional level but allow additional should use plain language if needed instead supports so that learners can appropriately of reverting to content from earlier grades express and receive information (see (for example, plain language version of Section 5.1 for examples). With accommo- Romeo and Juliet instead of toddler books). dations, all students engage with the same Table 4 provides a brief overview of the construct, but the timing, the way students differences between accommodations and respond to assessment items, or the way modifications at three stages of assessment items are presented may be adapted development. This table also presents for individuals (Bolt and Thurlow 2004). information on how UDA can help mitigate Accommodations exist to remove barriers some of the challenges associated with and provide more equitable opportunities accommodations and modifications. Having a universally designed assessment will minimize, but not fully eliminate, the need for accommodations. Even with UDA, many students will still require a full range of accommodations to accurately demonstrate their knowledge. Using Principles of UDA to Design 24 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications TABLE 4: Accommodations and Modifications during Assessment Design, Implementation, and Scoring Stage Accommodations Modifications UDA Considerations Assessment design Understand that Content or constructs Consider barriers students may need are changed to provide such as those found individualized students information in instructions, accommodations to at different levels of problem-solving, and access assessments, complexity or difficulty. response and attempt but these will not to remove them during change the constructs assessment design. of the assessment. Examine content for bias. Assessment Before implementing Before administering Universally designed implementation assessments, student modified assessments, assessments may accommodations students who will take reduce the need for should be identified modified assessments accommodations and administrators should be identified. or modifications by should be made aware providing students of individualized with options for accommodations on representation, assessments. response, and engagement. Scoring Accommodations do Because modified Universally designed not change tested assessments test assessments do not constructs or make the different constructs, modify constructs test content easier, so scores will need to be therefore can be any assessment taken flagged so that they scored like any other with accommodations are not conflated with assessment but should be scored and standard assessments. should disaggregate treated like all other data to highlight assessments. possible inequalities in education systems. Using Principles of UDA to Design 25 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications According to United States-Based Research... • In the United States, the eight most frequently provided accommodations are (1) extended time, (2) small group/individual administration, (3) test items read aloud, (4) directions read aloud, (5) alternative setting, (6) clarification of directions, (7) preferred seating, and (8) breaks as needed (Lazarus, Thompson, and Thurlow 2006). • Allowing students to use accommodations, such as computer administration, oral presentation, and extended time, provides them with more accurate ways of demonstrating their knowledge and abilities, often leading to increased student test scores (Thompson and Thurlow 2002). • Usage of assistive technology devices in instruction and assessment has grown steadily over the past decade, but Bouck and Long (2021) suggest that more pro- fessional development is needed to familiarize schools with ways in which devices can support learning and assessment practice. 5.1. Understanding Accommodations Accommodations are considered to be changes to the external features of the test (for example, how long the test takes and the setting) for individuals, while the content of what is being assessed remains the same (Dembitzer and Kettler 2018). These supports should respond to a student’s learning strengths and needs. Accommodations can be both formal (decided through a comprehensive evaluation and clarified in a student’s individualized education plan [IEP]) and informal. Accommodations can also occur in instruction and assessment and are not designed to give learners with disabilities an unfair advantage but instead reduce barriers and allow students to effectively receive information and express learning. Accommodations are generally organized into five categories: presentation, response, timing and scheduling, setting, and linguistic. Examples of common accommoda- tions for assessments are listed by category in Figure 1 (Lazarus, Thompson, and Thurlow 2006). Using Principles of UDA to Design 26 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications FIGURE 1: Assessment Accommodations Linguistic Accomodations Presentation Response Setting Time/Scheduling Linguistic Accomodations Accomodations Accomodations Accomodations Accomodations Follow the UDL Follow the UDL Students to take Follow all three Follow the two principle of means principle of means assessments in UDL principles by UDL principles of representation of expression to different locations, allowing teachers (expression and Explanation and allows students share knowledge follows two to selec times when reception) by who access in different ways UDL principles students are most lessoning the assessments in including assistive (expression motivated to learn language load of a different modalities. devices, graphics, and reception). and also facilitates test and ensuring and manipulative. Provides alternative test taking over thata assessments environment to a period of time are provided in the express information allowing for breaks most appropriate confidentially and and extra time to language such as receive information process. sign language. with less distractions. • Large print • Braille • Change in • Extended time • Receive instruction • Braille • Scribe setting to reduce allowing frequent and provide • Sign language • Augmentative distraction to breaks responses using • Tactile graphics and Assistive and from other • Change in the sign language • Human readers Communication students schedule to allow • Oral interpretation Examples • Computerized (AAC) • Change in setting for ideal testing of test items in the Information • Calculator to ensure physical time native language including screen • Spelling and access • Bilingual readers grammar devices dictionary or glossary Learners with... Print disabilities Physical disabilities, Physical disabilities, Physical disabilities, Deaf/hard of (e.g. dyslexia), blind/low vision, blind/low vision, learning disabilities, hearing, linguistic Learners blind/low vision, learning, disabilities, who use AAC attentional or minorities linguistic minorities intellectual disability behavioral disabilities, (challenges intellectual with memory or disabillities sequencing) Source: Adapted from Lazarus, Thompson, and Thurlow (2006). Using Principles of UDA to Design 27 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications • With all the choices available for accom- modations, it may be challenging for To ensure fairness, it is important teachers to know which accommodations that accommodations DO NOT: to use. When selecting accommodations, teachers should consider the following • Coach students during testing or questions: provide answers, | What are the student’s learning • Edit students’ work, strengths and needs? • Allow students to answer fewer | How do the student’s learning needs questions (that would be a affect the achievement of the grade- modification), level content standards? • Give clues to the answer, and | What specialized instruction does the student need to achieve grade-level • Change content by paraprashing content standards? or offering additional infomration (Cortiella 2005). | Do the accommodations being offered change the construct of the assessment in any way? (See Section 5.3) | What barriers are created by the way thev assessment is set up, and how might accommodations reduce barriers for students with disabilities? Once accommodations are offered, teachers and students should ideally discuss the accom- modations used and which ones were most helpful to evaluate and which accommodations should be continued or discontinued in the future. As noted above, accommodations are considerations for accessibity or specific supports that are different from person to person, depending on the type of disability and other factors (World Bank 2021). All accommoda- tions should be decided before the assessment, and the student should be familiar with the changes (it is not helpful to introduce new accommodations for the first time on the day of an assessment). Annex D provides suggestions for accommodations that may be helpful for students who are blind or have low vision, are deaf or hard of hearing, have complex disabil- ities, or are deafblind. When providing accommodations, it is important that teachers realize that this does not give unfair advantages to learners with disabilities but instead removes barriers that may limit how they can express their knowledge. Using Principles of UDA to Design 28 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications Keeping high standards for modified tests maintains high expectations for students with disabilities. Accommodation decisions, whether for instruction or assessment, are most appropriate when they are evidence based. This evidence could include observations from teachers, testimonials from students, information from documents like IEPs, and parent inputs. The evidence inputs may vary by context, but in any context it is important to consider that accommodation deci- sions should be carefully considered and based on available documentation and evidence. 5.2. Understanding Modifications Modifying an assessment signifies changes to the instruction given or content of the assess- ment. Modifications are often provided to learners with developmental disabilities, complex support needs, or intellectual disability. An example of a modification includes changing the test content or answers to allow a simplified assessment of constructs (Hamilton and Kessler, n.d.). Other examples of modifications include the following (Dembitzer and Kettler 2018): • Grading based on pass/fail and/or completion of work instead of content. • Testing the student on easier content compared to other students in the classroom. For example, if most students are learning two-digit addition (35 + 57), a modification could be testing on single-digit addition (4 + 8). • Reducing multiple choice options. • Grading on a different standard (for example, students get credit for completing tasks instead of grading the student’s work). In practice, assessment modifications often go further than the bullet points above, and modified assessments may test some students on completely different content or skills. The challenge in allowing students with intellectual disability or other learners with disabilities to have modified tests is that the results of the assessment are no longer comparable with students who have completed non-modified tests (that is, the norm-reference is lost). Modified tests can only be used to track progress of an individual or others who may have the same modified tests, which makes comparability of learning gains difficult. Furthermore, it is important to note that the consistent use of modifications can “increase the gap between the achievement of students with disabilities and grade level expectations” (s2). Use of modifications may negatively affect a student’s educational career and ability to progress. Using Principles of UDA to Design 29 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications If modifications are set arbitrarily and/or consistently too low—which is often the case for learners with intellectual disability—they can reduce learning and the opportunity for growth. If modifications are deemed necessary, modified tests should maintain the objective of assessing learning progress toward the same academic standards that are expected of all students. For example, if a student with an intellectual disability is provided 5 vocabulary questions instead of 20 but from the same vocabulary being tested, this would be an example of an adaption. However, a modification would entail selecting easier vocabulary options that are considered to be at a more basic level than the ones being provided to the other students. Modifications for Learners with Intellectual Disability Learners with intellectual disability may require modifications for formative and summative assessments, especially as content becomes more challenging. Most assessments discussed in this toolkit are normative, where assessments are used to compare learning of students at a similar acedemic level. However, for learners with intellectual disability, criterion-based assessments, which look at individual-based performance and their individual growth, may be more appropriate. This means tai- loring tests and content to the academic level of the students to ensure the content is appropriate while still moving the students forward in their academic growth. When providing modifications for learners with intellectual disability, it is important to • Individualize assessment. Not all modifications should be given to all learners with intellectual disability as each child has its own unique skills and will progress at its own rate. • Align with the curriculum. Although not all learners with intellectual disability will be able to perform at the grade level, students with intellectual disability should have access to literacy and numeracy and be introduced to other content such as science and social studies that is at the right level. The best way to avoid introducing modifications that change the constructs of assessments is to have clear communication about the objectives of the assessment (AERA, APA, and NCME 2014) and, where standards are available, to align expectations of assessments with the curriculum or standards (Martone and Sireci 2009). It should be noted that teachers often adjust standards, expectations, and difficulty of tasks in inclusive classrooms to allow for inclusive participation, but these practices may interfere with how student scores can be interpreted in large-scale assessments. Using Principles of UDA to Design 30 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications Standardized Tests and Modifications Some countries—such as the United States and Australia—have introduced legislation that requires learners with disabilities to have access to the general education curric- ulum as well as take part in standardized testing.9 The premise is that using and being tested on grade-level standards will ensure high expectations for learners with intel- lectual disability and, thus, further promote access to the curriculum (Davies 2018). The intent of standardized tests is to assess the extent to which educational standards are being met. These tests are designed for system accountability, and disaggregated data are used by governments to ensure all students are making learning progress. In general, standardized tests are unable to be modified given the complexity of the tests and the various subtests that are being administered. In other words, modified tests do not provide comparable data to standardized tests due to the changes made to the constructs of the test. Accommodations, however, can be provided as long as they do not change the construct that is being tested (see below for more details). 5.3. Accommodations versus Modifications Accommodations and modifications are similar in many ways, including the following (National Center for Learning Disabilities, n.d.):9 • Both allow students to receive and express information differently than standard expec- tations to accurately gauge learning and progress. • Both can be applied to classroom instruction; applied to formative, summative, and standardized testing; and used across all subjects. • Both should be determined in advance of testing and are often referenced in a student’s IEP; determination of supports should be decided by the teacher, student, and/or caregivers. • Both should be individualized to the specific student and not based on a disability cate- gory, grade level, or instructional setting. 9 It should be noted that the United States allows for 1 percent of students who are considered to have significant cognitive disabilities to take alternative assessments. Other countries, such as Australia, require all students regardless of support needs or cognitive level to participate in standardized testing. Using Principles of UDA to Design 31 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications • Both should consider a student’s learning strengths and learning needs. • Both are often mandated within the country’s education policies and legislations. Accommodations and modifications are different in the following ways: • Content. Modifications deliberately decrease the difficulty of an assessment’s content while accommodations provide the same content but allow students to express learning in different ways. • Identified disability. Teachers can provide accommodations to any student who is struggling if they think it will support the student in better receiving information and expressing knowledge, regardless of whether the student is identified as having a disability. This is particularly helpful in countries with emerging disability identification systems and practices, where there could be many students who struggle with assess- ments due to an unidentified disability or other reasons. Modifications should only be used sparingly as part of an IEP and only for those students who have previously been identified as eligible for special education services. • Consent. A teacher can provide accommodations informally for individuals (allowing for additional breaks and daily check-ins to make sure the student understands the lessons and tasks) as well as formally through an IEP process. As modifications reduce or change the difficulty level of content, modifications should only be applied with full and informed parental consent and should be a part of an IEP. • Comparability. Accommodations do not change the core content of instruction or testing, and students tested with accommodations will have comparable data to other students. However, when modifications are made to a test, students who take these assessments can no longer be compared with students who took an unmodified test. Results of modified tests are only used to compare individual student progress over time. • Usage and caution. Individualized accommodations should be used as needed for all students who require them. Given the challenges of reduced learning outcomes, modifications should be used sparingly and only when students have shown that they cannot be successful taking assessments with the same content as their peers even with accommodations. Using Principles of UDA to Design 32 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications Accommodations and modifications are similar yet different types of supports. Table 5 pro- vides examples of how these supports are slightly different. TABLE 5: Accommodations versus Modifications Assessment Accommodation Modification Classroom Methods in which learners take the test Testing may cover less material or assessment and express information are varied and/ address less complex content. or supported to reduce possible barriers related to disabilities. Standardized The same tests are provided Students take a reduced level of test or an assessment to all students with a variety of alternative test, which is often not allowed accommodations. in many countries. Other countries have strong restrictions on how many students can take alternative testing as a means to promote student access to the curriculum as much as possible. Example 1 A student who has challenges writing A student who has challenges writing answers open-ended questions verbally receives multiple response options and instead of writing the answer. The points at the correct answer instead construct of ‘answer the question without of writing the response to open-ended prompts’ remains the same. questions. The construct changes because the student has a selection of 3 answers instead of an open-ended question. Example 2 A student who is blind takes that same A student who is blind takes a different test but has the test provided in braille reading test in braille with easier and responds in braille or provides sentence structures and vocabulary. oral answers. The constructs have not This practice is not recommened as the changed, but the manner of accessing construct was changed instead of the the printed text has. means of accessing information. Notably, if the construct of an assessment is changed by providing an accommodation, this may result in a modification. For example, although the accommodation of reading test questions aloud is allowable for most science and math tests, if the construct focuses on a student visually or tactilely reading a passage independently, a read-aloud accommodation would change the construct of the test and, thus, be a modification. Similarly, extended time is an accommodation used in many assessments if the construct is a response within a timed period. For these reasons, it is critically important to identify the desired construct of an assessment so that accommodations do not become modifications. Using Principles of UDA to Design 33 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications 5.4. Ensuring Proficiency in Alternate Formats A student should never encounter an assessment format or accommodation for the first time on the day the assessment is used. Whether the assessment is in an alternate format (for example, braille or large print) or a student has access to an accommodation (for example, an assistive technology device), assessments are not a time to experiment. Rather, any alter- nate format or accommodation provided to the student should be decided upon in advance, with ample time for the student to become familiar with the format or accommodation. Ideally, students will use the same formats and accommodations in assessments that they use in their day-to-day instruction. In summary, as Figure 2 indicates, most students benefit from universally designed assess- ments that have accessibility features; some students, even with a UDA test, will require accommodations, and a few will require modifications. FIGURE 2: Accommodations versus Modifications Summary 5.5. Environmental Considerations The environment in which students take assessments can have a significant impact on their performance. Background noise, lighting, glare, and visual clutter can affect a student’s ability to perform at the optimal level. For example, a student with attentional disabilities may be distracted by background noise. A student who has low vision and light sensitivity may be more sensitive to glare from a nearby window that affects the student’s ability to see the test paper. Therefore, it is critical to provide visual and environmental information to students before the assessment. Orienting students to the testing room, describing the items on and around their desk, and showing them where the washroom is located should be standard practice during any testing situation. Table 6 provides environmental considerations that help all students including those students with disabilities. Using Principles of UDA to Design 34 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications TABLE 6: Environment Considerations that Support UDA Make sure there is adequate lighting in the room, and ask students Lighting if they need additional task lighting. Check if a window or overhead lighting causes glare or if students are positioned in a place where direct sunlight affects their vision. Sunlight and glare If using computers or tablets, make sure the screens do not have a glare. Ensure that students can move themselves closer or farther Ability to adjust away from the paper or computer screen to better view or access positioning materials on the desk. Visual background Review the testing space to make sure it is free of visual clutter and clutter other visual distractions as much as possible. Arrange test materials in consistent places so that learners with disabilities know where items are located and can access them; Access to materials remove possible physical barriers between materials and students to faciliate access. A test environment with limited background noise is best for all students. Note that some noises that may not be noticed by others, Quiet environment such as a fan, outside gardening equipment, or noise from another room, may need even quieter environments to focus. Test in an environment familiar to the student to reduce stress. Make sure students who are blind or deafblind receive information Familiar about the environment around them, including who is in the room, Environment how the room is set up, and what is happening in the room. They should also have access to all communication that is intended for the group. Using Principles of UDA to Design 35 Toolkit Accessible Learning Assessments Understanding Accommodations and Modifications Photo Credit: Leonard Cheshire, UK 6. Understanding Accessibility This section reviews... • General features of accessibility, including visual and communication • Accessibility features for chalkboard-administered assessments • Accessibility features for printed-text assessments • Accessibility features for technology-based assessments • Additional accessibility features for learners who are blind or have low vision or who are deaf or hard of hearing 6.1. General Accessibility Features Too often tests are designed without considering the accessibility needs of different students. Although accessible versions of assessments can be provided to students who require them, many countries do not have the required systems to appropriately identify students’ needs and, thus, present them with alternative versions of assessments. When all Using Principles of UDA to Design 36 Toolkit Accessible Learning Assessments Understanding Accessibility students are provided assessments with accessiblity features, students with a variety of needs will benefit even if they have not yet been officially identified as having those needs. Assessments in LMICs are delivered through various means, including chalkboards, paper tests, tablets, or technology based on the country’s access to resources and materials. This section provides general guidelines on how to develop assessments with different types of accessibility features to support visual access, communication, and focus. The section also provides general accessibility guidelines that can be used regardless of how the assessment is delivered. Understanding Functional Vision and Hearing Having information about a student’s sensory status is critical to effective teaching, learning, and assessment. Clinical assessments are typically conducted by vision and hearing professionals, including ophthalmologists, low-vision specialists, and audiologists. These professionals provide clinical diagnostic information about how well a student can see or hear. Functional or informal vision and hearing assessments are typically conducted by someone who can observe the student in natural settings, such as in the classroom, at home, or on the playground. The information from both a clinical and functional vision and hearing assessment can provide more accurate information about what a student is able to see and hear and what supports the stu- dent will need to ensure accessibility. However, these services may be limited within an LMIC context. It should not be assumed that if a student is considered ‘blind’, the student does not have some usable vision. Legal blindness is a spectrum of vision loss that includes individuals who have low vision and those who have no residual vision. Therefore, some students who are identified as being blind may not use braille but may instead rely on large print. Additionally, students with a progressive vision loss may be transi- tioning from large print to braille and may need both accommodations to fully access the material. The same holds true for students who are identified as deaf. Some students who are deaf use hearing aids or assistive devices to maximize their residual hearing while also relying on a sign language interpreter to access communication. In alignment with UDL research, almost all educational activities (including assessment) require some form of representation, response, and student engagement. In assessments, representation often refers to how students access the required task. How information is presented to students matters; if students cannot understand what is required of them, they will not succeed. Similarly, all assessments require students to respond by demonstrating Using Principles of UDA to Design 37 Toolkit Accessible Learning Assessments Understanding Accessibility knowledge or skills. If the required format for response is difficult for a student, the teacher may receive inaccurate information about what a student knows versus what a student does not know. Finally, assessment content and requirements can either be motivating or demo- tivating (or even offensive) to students. If a student is not motivated to succeed, information from assessments may be invalid. This section provides ‘how to’ information for getting started on accessible assessments. Whether students take a classroom, technology-based, or other type of assessment, this section provides information on how to include accessibility features. An Accessibility Checklist can also be found in Annex C. 6.1.1. Visual Presentation Accessibility Features Certain steps can be taken to allow students with a wide range of visual function to access standard assessments. Having a clear assessment presentation is crucial as it supports a variety of learners including students who have low vision, learning disabilities, or other challenges visually processing information. It may also help students who are visual learners, as the assessment’s presentation may make content become more relatable. For students who are blind or have low vision, additional considerations are needed to ensure that assessments are accessible. Below are general accessibility recommendations for students who are blind or have low vision, while additional accessibility suggestions for students who access print but have low vision can be found in Section 7.5. Recommendations for Visual Accessibility • Provide both auditory and visual cues for assessment items. • Avoid items that require knowledge of specific visual information (for example, color and references to light/dark). • Allow the use of magnifying glasses or other assistive devices which may be needed for learners who have low vision. • For items that require looking at a screen or document, ensure that there is high con- trast; that letters, symbols, and numbers are at least 14-point font; and that there is a large amount of white space on the page or screen. Using Principles of UDA to Design 38 Toolkit Accessible Learning Assessments Understanding Accessibility Communication Presentation Accessibility Features 6.1.2. Assessments that rely solely on verbal communication to explain concepts or directions or express knowledge can create barriers for many students who have challenges hearing instructions or expressing their knowledge verbally. For example, students who are deaf or hard of hearing may face barriers in assessments when items are presented in verbal formats only or do not have corresponding visual inputs to help these students understand requirements. Test designers may also create barriers if students must engage with the assessment through audiological inputs or listening comprehension tasks. Students who are deaf, hard of hearing, or have audio processing disorder and second-language learners may need more time to process information auditorily than their peers who are hearing or may need information provided to them in their first language—sign language (Rose et al. 2008). In either case, quickly presented verbal information (such as instructions read aloud before an assessment or verbally presented assessment items) may disadvantage many different learners and create a comprehension barrier. Likewise, students with communication-related disabilities, including those who have chal- lenges using traditional speech, use AAC, or are not fluent in the language of instruction, may have challenges expressing information. In some cases, students may express the correct information verbally but have challenges with articulation that may make it difficult for teachers or assessors to understand the student. The following suggested accessibility features can reduce communication barriers. Recommendations for Communication Accessibility • Give clear step-by-step instructions, and present each instruction only as needed to avoid students getting confused or overwhelmed. For example, the following instruction can be provided as “1) read the paragraph 2) answer the questions at the bottom of the page” instead of “read the paragraph and then read the instructions at the end of the page.” • Provide clear objectives and instructions. Instructions should be presented visually using text and/or pictures as well as explained orally. • Use clear and user-friendly language and avoid using complicated terminology. • Use consistent language throughout. For example, do not use ‘ball’ in one sentence and ‘soccer ball’ in the second sentence to refer to the same object. Using Principles of UDA to Design 39 Toolkit Accessible Learning Assessments Understanding Accessibility • Provide the assessment in students’ first language, including allowing assessments to be completed in sign language. • Ensure that the assessment does not require understanding specific sounds to address item constructs, for example, asking a learner what insect makes a buzz sound. • Avoid items that may be an external experience to students who are deaf or hard of hearing (phone calls, specific sound references). • Allow assistive devices, such as sound amplifications systems, AAC, or communication boards. • Allow students to respond to questions in a variety of ways including orally and written and with assistive devices. 6.1.3. Other Accessibility Features Many students may have challenges focusing on assessment content. Focusing challenges can be caused by intellectual disability, attentional disabilities (such as attention deficit hyperactivity disorder [ADHD]), prolonged hunger, or exposure to trauma. Therefore, tests can be designed in a way that helps increase motivation as well as promote focus. Recommendations for Other Accessibility Features • Allow untimed tests if it is not a part of the test construct. • Design assessments that allow multisensory engagement, including manipulatives that can be used by all students (if they do not violate the construct tested). • Include visuals, images, or icons to reinforce concepts. • Design assessments so that breaks take place as part of the testing process. Using Principles of UDA to Design 40 Toolkit Accessible Learning Assessments Understanding Accessibility 6.2. Accessibility Features for Chalkboard-Administered Assessments Chalkboard-administered assessments may put students at risk of not grasping information for several reasons. In chalkboard-administered assessments, students must listen to a teacher explain an assessment topic, read the topic off a blackboard, and often copy it to paper. During each of these testing procedures, students may miss important information and, thus, not be able to demonstrate their full abilities in the assessment. In addition, stu- dents who are blind or have low vision will need additional time to avoid such situations. Recommendations for Chalkboard Accessibility • Ensure the chalkboard is cleaned before writing on it. A clean chalkboard enhances contrast between the board and the information written on it, thus increasing legibility. • Ensure items on the board are printed in a large, legible print. • Ensure students who are blind or have low vision have access to a reader and/or scribe if needed. • When explaining a particular item, be sure to point to that item so that students can focus on what is needed to be successful. • When explaining an item, be sure to use a clear voice and language that students can understand in the class (unless the construct of the test dictates otherwise). • If explaining an item, ensure simultaneous sign language is available if there are stu- dents who are deaf in the classroom. • If instructions are provided to students in sign language, ensure the teacher or inter- preter points to the particular item on which students should focus. • If possible, provide students with copies of the assessment, so that they do not make mistakes copying from the board. • If it is not possible for students to receive copies of the assessment, allow students to help one another copy test items from the board to ensure accuracy and not create barriers for students with fine motor or other writing difficulties. • If written responses are required, allow accommodations for verbal responses. • Provide students additional time to use alternate formats and processes. Using Principles of UDA to Design 41 Toolkit Accessible Learning Assessments Understanding Accessibility The abovementioned points align with the UDL principles of “multiple means of representa- tion, engagement, and response” outlined by the Center for Accessible Special Technologies (CAST) (2018). 6.3. Accessibility Features for Printed Text Standard, print-based assessments create barriers in comprehension for students who are blind, have low vision, or have other challenges with traditional print, including students with several forms of learning disabilities. All assessment should be visually accessible. Recommendations for Printed Text Accessibility Use 18-point font or larger. Provide high-contrast materials. Ideally, assessments should also be printed in both 24-point and 26-point fonts to allow learners to choose the size that is most comfortable for them. Ensure adequate white space Ensure leading (space between on the page, including margins lines of text) is 1.25 spaces or of 1 inch or more. greater. Using Principles of UDA to Design 42 Toolkit Accessible Learning Assessments Understanding Accessibility Locate text and relevant Ensure at least 1-inch margins diagrams near each other on on all sides of the printed text. pages. Arial Futura Helvetica Rockwell Clarendon Courier Use sans serif fonts (for example, Arial, Futura, and Helvetica) or block-serif fonts (for example, Rockwell, Clarendon, Serifa, Courier, and Memphis). In addition, sans serif, monospaced, and roman fonts have also been found helpful for learners with dyslexia while italic fonts are often more difficult for them (Rello and Baeza-Yates 2013). Symbol versus Alphabet Text The print size of nonalphabetic languages may also affect a student’s ability to access information because of the detailed symbols required to read words, also known as complexity. For example, Arabic is two times smaller than English when printed from standard fonts on a computer. The print size chosen must consider the threshold of the smallest letter or symbol used in that language. Just because a standard large print may be 24-point font, it does not mean the smallest symbol will be at that font size. Using Principles of UDA to Design 43 Toolkit Accessible Learning Assessments Understanding Accessibility 6.4. Accessibility Features for Technology-Based Assessments Technology-based tests are becoming more common and inclusive of students with dis- abilities, as they can reduce the need for many commonly used accommodations, such as human readers, large print, braille, and scribes, for students who are blind or have low vision or who are deaf or hard of hearing. Some of these tests will have accessibility features built into the platform. Students can also use accessibility software programs, such as screen readers, accessible calculators, and keyboards. An advantage of using technology-based tests is that students can control the font size, the speed of audio output, the color and size of the mouse pointer, and the contrast. If a test is computer generated, students need to have experience with the accessibility software and test platform they will use before the assessment. Many variables need to be considered, including what accommodations are built into the test and what technologies need to be compatible with the student’s own adaptive technology. For example, if the student uses a screen reader, the test platform needs to be compatible with that particular screen reader. It is imperative that these compat- ibilities are sorted out before administering the test. Technology-based assessments provide far more opportunities for adaptability in assess- ments than paper and pencil tests. Within technology-based assessments, students can • Adjust the volume of audio inputs if they are wearing headphones or ear buds, • Adjust the size of visual stimuli to be larger or smaller as needed, • Adjust the contrast to reduce glare and provide sharper imagery, • Use selection features such as highlighting text for emphasis, and • Use built-in tools such as calculators or dictionaries. However, whenever options and features are introduced on a technology-based assessment, students should have either had an opportunity to practice such features as part of their everyday classroom activities or, at least, have had an opportunity to practice extensively on the testing platform before the day of the assessment. In addition to ensuring that accessibil- ity features are familiar to students, test designers should also ensure the following: • ‘Scripted’ text (that is, computer-generated reading of test items) is consistent, the information is easily understandable, and it is spoken in a language that students under- stand (Johnstone, Higgins, and Fedorchak 2019). Using Principles of UDA to Design 44 Toolkit Accessible Learning Assessments Understanding Accessibility • If test items are read to students, simultaneous sign language interpretation should be available, either on screen within the assessment itself or from an interpreter. • If items are read to students, captions should be available. • Item boxes in which students must respond are clearly marked and easy for an asses- sor or student to input answers. • ‘Forward’ and ’back’ buttons are easily recognizable and placed at the bottom cor- ners of items. • Headings should be hyperlinked and follow a consistent format. Numbering of head- ings is preferred compared to only making changes in color and font size which may be less noticeable to persons who are blind or have low vision. It is important to note that not all LMICs have access to educational technologies. If technol- ogies do exist within the country, there may not be equitable access to them. For example, learners in more remote or rural areas or areas with high levels of poverty may not have access to technology. Using Principles of UDA to Design 45 Toolkit Accessible Learning Assessments Understanding Accessibility Photo Credit: ShutterStock 7. Universally Designed Classroom Assessments (Formative and Summative Assessments) This section reviews... • An overview of classroom assessments • Features of universally designed classroom assessments 7.1. Understanding Classroom Assessments Classroom assessments can be characterized as assessments ‘as’ or ‘for’ learning. As a result, the main focus of these assessments is to provide feedback to teachers and learners to support teaching and learning in classrooms (Clarke and Luna-Bazaldua 2021). However, Using Principles of UDA to Design 46 Toolkit Accessible Learning Assessments Universally Designed Classroom Assessments UDA and Giving Student Feedback The purpose of both formative and summative assessments is to improve student learning and shift teaching strategies to ensure that students are learning. Giving students feedback on what they are doing right and areas in which they can improve is extremely important. UDA principles can be applied to a wide range of assess- ments, from formative to summative. UDL guidelines can also be used in classrooms to provide feedback to students. For example, • Multiple means of representation. Teachers and students alike can provide written, gestural, or verbal feedback to help guide students’ understanding. • Multiple means of action and response. Students can receive feedback and inputs from fellow students in paired, small group, or large group exercises. • Multiple means of engagement. Students can improve motivation by learning what forms of feedback are most motivating to students—whether the feedback should be written, verbal, artifactual (for example, stickers or other similar objects), or gestural (for example, thumbs up). the nature of classroom assessments • Formative assessment. The goal of a does not mean that they are low stakes. formative assessment is to gauge how Classroom assessments can affect school a student is progressing with the new decisions about student placement, pro- content and adapt teaching approaches gression to the next grade, and a teacher’s as needed. Examples include quizzes, approach for teaching individual students. observations, homework assignments, If a teacher does not have an accurate pic- class discussions, and so on. ture of students’ knowledge, it might affect what is taught, how it is taught, and the • Summative assessment. The goal of teacher’s overall perspective on the learn- a summative assessment is to evaluate ing abilities of certain students. Usually, a student’s learning at the end of an classroom assessments are formative and instructional unit and compare it to summative. standards or benchmarks. Examples include end-of-the unit exams and national assessments. Using Principles of UDA to Design 47 Toolkit Accessible Learning Assessments Universally Designed Classroom Assessments 7.2. Universally Designing glimpse into their knowledge. Journals Classroom Assessment can also be visual with drawings or even collecting items from their sur- Features roundings to express themselves. UDA in classroom assessments can be • Curriculum-based measurement implemented in ways similar to UDL (CBM). It refers to continuous assess- because such assessments are often not ments that students take related to the bound by the constraints of standardization current content they are learning. These often found in large-scale assessments. assessments are a way for teachers to Instead, teachers can focus on the primary continuously check progress. CBM may goal of assessment: understanding what be a short math quiz or a read-aloud students learned. Examples of classroom activity that occurs in class. assessments that can be used in high- or low-technology-rich environments are listed • Authentic assessments. Students can below. demonstrate what they have learned through solving real-life problems that • Verbal, signed, or other responses. require them to apply their academic Teachers can quickly gauge if students learning. understand by asking questions and asking for verbal or signed responses. • Presentations and products. Students Higher-technology options for quick can demonstrate learning through pre- responses from students include senting their knowledge to others in the ‘clickers’ that count student responses class or developing creative products or low-technology options such as (adapted from Salend 2009). response cards that students can hold up in the air. Each of these forms of assessment may be engaging to some students but demotivat- • Observation through diaries/jour- ing to others. Therefore, teachers should nals. Students can write about their use different modes of assessment so that thoughts in relation to phenomena they they can best understand what and how are observing in the world or content students have learned in their classrooms. they are learning in the classroom. However, each of these assessments can A student’s writings can provide a introduce barriers to students. Using Principles of UDA to Design 48 Toolkit Accessible Learning Assessments Universally Designed Classroom Assessments Roles of Different Stakeholders to Support UDA in Classrooms National assessment authorities: • Pilot inclusive formative assessment or large-scale assessment that use UDA and record shifts in test scores between traditional and inclusive assessments and act on results. • Commit to exploring UDA as means to better demonstrate student knowledge. • Define test purpose and approach and constructs so that constructs can remain with additional inclusive practices. • Train teachers and school staff on the value of UDA and inclusive practices. School leadership: • Support using UDA in formative and summative assessments. • Develop policies that allow students to use different accommodations during assessment periods. • For any IEPs, clarify what type of accommodations should be provided to stu- dents and if modifications are required how education will be aligned with the curriculum. Teachers: • Allow students to use accommodations during assessments and review which accommodations are most useful for the student. • Use data from assessments to determine which students may be continuing to struggle and provide small group instruction. Using Principles of UDA to Design 49 Toolkit Accessible Learning Assessments Universally Designed Classroom Assessments Photo Credit: World Bank 8. Universally Designed EGRA and EGMA This section reviews... • The introductory steps to implementing UDA within EGRA and EGMA, including defining constructs, designing for accessibility, providing accommodations, pilot- ing, and analyzing information • That concepts may be used by national governments administering their own tests or using EGRA and EGMA Using Principles of UDA to Design 50 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA 8.1. Understanding Large-Scale Assessments As mentioned previously, the goal of UDA is to assess all learners using the same or similar test materials to have norm-referenced comparability data for a target population (AERA, APA, and NCME 2014). However, in the early 2000s, many countries were implementing national standardized assessments that did not capture what students in the country knew. Students scored so poorly that the test could not identify their current knowledge and skills (RTI International 2016). As a result, in 2006, USAID developed EGRA, with the goal of developing an instrument that could accurately measure how well primary grade students in LMICs were acquiring reading skills (RTI International 2016). EGRA is primarily used by early grade reading projects and, as of 2015, had been used by over 30 organizations in more than 70 countries. Once EGRA had been widely used, a similar disconnect was addressed in assessing knowl- edge in mathematics. In 2008, EGMA was developed to accurately measure how students in the same countries were acquiring mathematics skills. As of 2016, this assessment had been used in 22 countries around the world (RTI International 2016). Recently, many organizations have tried to adapt or modify EGRA and EGMA to be inclusive of learners with disabilities with varying levels of success. Given this interest in ensuring that EGRA and EGMA are inclusive of learners with disabilities, this toolkit provides particular attention to these two forms of assessments. Furthermore, EGMA was selected for the Tajikistan UDA pilot and thus has additional relevance to this toolkit. 8.2. Universally Designing EGRA and EGMA The 2014 EGMA toolkit provides an overview of the key domains and administration pro- cedures of EGMA. The suggested UDA changes presented in Table 7 can likely be made to the assessment without changing the core domains and competencies being tested. These changes align with UDL principles of multiple means of representation and response, which are discussed in further detail in Section 3.1 (CAST 2018). Furthermore, the examples from the Tajikistan UDA EGMA pilot and the adaptions were shared with the developers of the EGMA tool to ensure that the intended construct had not be changed as a result of the adaptation. This section provides an overview of adaptions for students who are blind and those who are deaf. For large-scale assessments, evidence-based practice does not rec- ommend modifying assessments for students with cognitive disabilities but instead provide accommodations as needed (see Section 5.2). This is, in fact, law in the United States as it is seen to promote access to the curriculum with few exceptions provided (Landau, Vohs, and Romano, n.d.). Using Principles of UDA to Design 51 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA Learners who are blind or have low vision may need additional accommodations for EGMA or large-scale math assessments including having tests in braille, 1.5 or 2 times more extended time, access to a large print calculator, images in larger sizes and high resolution, and ability to mark up or take notes on the test itself (Perkins School for the Blind, n.d.). Learners who are deaf and hard of hearing should have access to instructions and any questions provided in sign language (Cawthon and Leppo 2013). It is important to assess first that they have strong sign language skills so that they fully understand the instructions and content of the instructions or written tests. TABLE 7: Possible Adaptations for EGMA Subtests Subtest Adaption Description Number • Allow students to receive instructions This will ensure that students do not Identification in both written and oral formats. encounter linguistic or communication barriers in naming numbers. • Allow students to respond in their home language (including sign language) for all number identification items. • Allow students to write the number, say the number, or point to the number on a number line. Number • Allow verbal (spoken) or signed This will provide students with multiple Discrimination identification of numbers in number ways to see and hear the numbers they discrimination items, along with visual must discriminate from one another (that inputs for students. is, finding the larger number in a pair). • Allow ‘this one’ responses as students The current EGMA toolkit requires differentiate between the larger students to say the full name of the number in pairs rather than requiring larger number in a pair (for example, the them to say the name of the larger student says the number ‘732’ aloud), but number. this may introduce new and unnecessary memory requirements when the task is simply to identify the larger number in a pair or group. Addition and • Remove time limits on addition and Adding a time limit may introduce new Subtraction subtraction problems. In this domain, anxieties for students or may encourage the construct is the ability to add or them to rush through items, thus subtract. providing assessors with invalid data. Using Principles of UDA to Design 52 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA 8.3. Developing Literacy Assessments for Students Who Use Sign Language Attempting to retrofit existing learning assessments to include students who are deaf or hard of hearing often results in assessments that are less effective because they are not easily adapted. For learning to be adequately addressed, instruments must be developed consider- ing the unique learning and assessment needs of students who are deaf or hard of hearing. Below are three suggestions on how literacy assessments, including EGRA, can be designed to better include learners who are deaf or hard of hearing. A. Sign language assessments should include at least two tasks: vocabulary and language comprehension. Because language is critical to reading, a reading assessment that is appropriate and sufficiently sensitive to students who are deaf or hard of hearing should include language tasks that assess both vocabulary and language comprehension. For example, USAID Tusome Early Grade Reading Activity in Kenya assessed students who are deaf on four subtests (Piper et al. 2019). As sign language is not standardized in some countries, test development should be sensitive to regional variations in signs. Vocabulary. Vocabulary knowledge can be assessed receptively (students select the picture that matches the sign) and/or expressively (students label the pic- tures). For both tests, the goals are to include sufficiently challenging vocabulary to be sensitive to the range of abilities possessed by students who are deaf or hard of hearing and to select stimuli that do not allow students to guess the answer due to sign iconicity. Selection of words should be based on language, not reading development. In other words, the selection of words on the vocabulary tasks should not be created based on text analyses as is typical when developing a literacy assessment instrument in a new context. Language comprehension. The typical literacy assessment language comprehen- sion task requires learners to answer questions about a story. A better task may be to present signed sentences individually to the learner (arranged in increasing complexity). For example, after watching a signed sentence, a learner who is deaf or hard of hearing can choose the matching picture out of four options. Using Principles of UDA to Design 53 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA B. Reading assessments should include at least three tasks: alphabetic knowledge, familiar word reading; and reading comprehension. Alphabetic knowledge. Knowledge of the letter names (not sounds) is a founda- tional skill for students who are deaf or hard of hearing. This task should require learners who are deaf or hard of hearing to exhaustively name the letters in a language. Familiar word reading. This skill should be measured by presenting individual words in order of complexity for students to read and provide the sign. Words that do not have a sign equivalent should be excluded. Reading comprehension. Subtasks should present simple print sentences and have the learner select the correct picture from an array of three. Subtasks should start with simple phrases and slowly become more complex. For each of these subtasks, it is critical for students who are deaf to understand what they are supposed to do. Visual cues may improve the validity of assessment results because students understand what is expected of them. Therefore, signed instructions, tactile/visual demonstrations by enumerators, and visual examples in the assessment itself may reduce construct-irrelevant errors (that is, those not related to students’ literacy skills) in the assessment. For the reading comprehension text, visuals that support comprehension of the passage can be part of all EGRA assessments. Such visuals should relate to infor- mation in passages but should not be merely decorative or distract readers from intended meaning of passages (Thompson and Thurlow 2002). C. Fluency should not be a component of literacy assessments for students who are deaf or hard of hearing. Fluency is typically embedded within literacy assessments. Fluency is, by its very nature, a measure of automaticity of reading knowledge, which is a more advanced skill than the ability to know the meaning of a printed word. Therefore, measuring flu- ency may add an unnecessary level of complexity to the assessment for students who are deaf or hard of hearing. Therefore, it may be better to only test knowledge (that is, what words are recognized) when assessing this population in the early grades. Stimuli should be arranged by difficulty and tested one at a time (Lazarus et al. 2022). Using Principles of UDA to Design 54 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA 8.4. Developing Literacy Assessments for Students who Use Braille When developing and administering literacy assessments for students who use braille, two important components need to be considered: (1) the student’s braille literacy skills and (2) the accessibility and accommodations available to students who prefer to use braille on literary assessments. Assessment of braille literacy. To support students’ ongoing acquisition of braille, it is essential to determine their fluency in both braille code skills and broader literacy skills. A meaningful assessment of braille literacy skills should include (1) a focus on literacy, not solely or primarily on braille code skills; (2) the use of a multifaceted approach; (3) an ongoing assessment over time; and (4) a meaningful integration of the assessment and the student’s instructional program. An assess- ment of braille literacy should include emergent literacy skills, formal literacy skills, and functional literacy skills (Region 4 Service Center 2015). Accessing standardized literacy assessments using braille. Research indicates that students using braille or large print to access standardized tests perform similarly to their peers without visual challenges who use standard print (Stone et al. 2010). This suggests that the use of braille (with appropriate accommodations) does not affect student performance or adversely affect the validity of the assess- ment. However, braille should only be used for assessment after it is determined that the student is fluent in decoding braille and only if the student uses braille reg- ularly for instructional purposes. The type of braille used for the assessment should match the version the student typically uses (contracted versus un-contracted). As discussed in other sections of the toolkit, when developing literacy assessments for students who are blind or have low vision, it is imperative to eliminate questions that use visual references (color, description of pictures, visual item identification) or that rely on visual recall. Accessibility features and accommodations. It is increasingly common to have tests that are designed and developed with braille readers in mind. Students can use screen readers for computer-generated tests with braille-ready files, including visual descriptions of pictures, headings for each section, and alt text, to ensure accessibility for braille users. The following accommodations are in addition, and/ or restated for their significance, to those indicated in Section 6.1.1 of the toolkit and should be considered for all students who use braille for literacy assessments. Using Principles of UDA to Design 55 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA • Test developers need to have extensive knowledge of braille formats. Braille versions of assessments need to be edited and reviewed for accuracy. • Additional time to take the test should be considered for all students using braille. Reading braille takes more time than visually reading print (Trent and Truan 1997). • Assessments should be printed on good quality braille paper or card stock and checked to make sure the braille is not worn down and is still legible. • One line of spacing should be between all lines of braille to avoid students ‘catching’ the dots from the line above. Guiding lines (continuous dot 6) should be placed on the appropriate subtasks (that is, vowel). • Response accommodations, such as presenting answers orally to a scribe who records on paper, using a braillewriter or slate and stylus, or using a computer word processing program, should be considered. 8.5. Steps for Implementing UDA Implementing UDA (within EGRA and EGMA) involves various steps. For any other large- scale assessment, the steps can be contextually modified. Step 1: Build consensus on the importance of UDA with key stakeholders. Global policies have shifted toward promoting inclusion in schools. As nations turn their focus to access and improved instructional practice, assessment is often overlooked as a site of inclusion. Honest and forthright discussions among stakeholders about inclusive assessment and the utility of UDA for improving accessibility are a useful first step. Stakeholders often agree about the concept of inclusion but disagree on the best ways to include all students. UDA presents one way of framing assessment design to align with inclusive ideals. Step 2: Build upon priority learning goals and standards to inform the constructs, target skills, and UDL strategies for an assessment. Nations often have standards or learning goals by grade level that can be used to inform what will be assessed and how stan- dards and content will be assessed (Clarke and Luna-Bazaldua 2021). Once the scope of an assessment is defined, another preliminary step is to identify the assessment’s target skills to ensure that the assessment is accessible and measures these skills accurately. Identifying both target and access skills is critical to promoting accessibility. Using Principles of UDA to Design 56 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA • Target skills are the tested constructs or what students must do to effectively demon- strate knowledge or skill on EGRA or EGMA. • Access skills are those skills that students must have to successfully comprehend and respond to the test items. A careful review of the target and access skills of any assessment constructs should be clearly articulated before developing an EGRA or EGMA subtest to ensure that the changes in design and administration do not affect the intended original constructs of the assess- ment. For example, EGMA requires students to listen to and respond to prompts in its Word Problems subtest. For this subtest, mathematical problem-solving is the target skill (what is being assessed), and listening comprehension is the access skill (what is needed to access the construct). Table 8 provides further examples of target and access skills frequently found in large-scale assessments. TABLE 8: Examples of Target and Access Skills Subject Area Target Skill Access Skill Reading Letter recognition (timed test) Swift processing of letter names Reading Fluency Spoken language Mathematics Number recognition (timed test) Swift processing of number names or quantities Mathematics Mathematical reasoning through word Reading skill, computation problems Source: Adapted from Kettler (2015). Step 3: Review questions and instructions for sensitivity and bias. All assessments might contain instructions, requirements, or examples that are either offensive or biased against particular populations within any given country or provide an unfair advantage to certain groups. Larger assessments (particularly national assessments, EGRAs, and EGMAs) should be shared with representatives from OPDs and other marginalized groups so that they can examine the assessments to identify and mediate potential biases and any item requirements that may be inaccessible, unfair, or unfamiliar to students with disabilities. For example, a student who is blind may not be able to identify an object based on color on a test. A student who is deaf or hard of hearing may not be able to answer a question about sounds that are alike or different. It is important that individuals who are deaf or hard Using Principles of UDA to Design 57 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA of hearing and individuals who are blind or have low vision are part of the test design and review process to make sure that information can be accurately accommodated through sign language and braille with no additional barriers. Having these individuals review items in advance may help stakeholders identify biases or assumptions in the existing assessment that can be adapted to allow more inclusive participation. Step 4: Follow accessible presentation guidelines. It is important to follow National Center of Educational Outcomes (NCEO) guidelines for universally designed assessments to design tests in a way that is accessible to students with a variety of needs. Section 6 provides more details that should be considered to improve the presentation of visual and communication accessibility features as well as other accessibility features. Step 5: Review materials and images for diversity and representation. One element of UDA is to include visual images, stories, or materials. When developing such materials, test designers need to ensure that characters and images represent the diversity within the country. In general, at least 50 percent of characters should be girls and 15 percent of char- acters should be individuals with disabilities. Although some assessments do not currently have imagery, as images begin to be used for accessibility purposes, they should follow these guidelines. These images and characters should represent individuals in empowering ways, not doing stereotypical tasks. For example, women should be seen as community leaders and entrepreneurs instead of only doing household tasks. For more guidance in this area, please consult ‘A Guide for Strengthening Gender Equality and Inclusiveness in Teacher and Learning Materials’ .10 Step 6: Conduct test reviews with content, assessment, and disability experts. Once a test is developed, it needs to be reviewed by experts. A formal review should be held with content area experts (such as experts in EGMA) to ensure that constructs were not changed, and nothing interferes with the original intent of the subtests. Second, a validation should be held with adults and professionals with disabilities (an OPD is ideal if possible) to ensure that no additional barriers can be removed. Most importantly, individuals who are deaf or hard of hearing and individuals who are blind or have low vision need to be part of the test design and review process to make sure that information can be accurately accommodated through sign language and braille. Step 7: Train assessors and teachers. Universally designed assessments may look dif- ferent or have slightly different procedures than assessments typically given to students. For this reason, it is important to develop training materials and then train assessors and 10 https://shared.rti.org/content/guide-promoting-gender-equality-and-inclusiveness-teaching-and-learning-materi- als-2#modal-29-628. Using Principles of UDA to Design 58 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA teachers on both the rationale (why a test is universally designed) and the procedural aspects of the assessment (how to administer it). Step 8: Pilot the UDA tool. Before administering an assessment at scale, it is important to pilot it with a diverse group of students. Ideally, this would also include piloting the assessment with students who have been previously identified as having disabilities, such as with students who are deaf or hard of hearing, are blind or have low vision, have intel- lectual disability, have learning disabilities, and so on, to determine if any barriers or biases have not yet been identified and eliminated. Target populations for accessibility features should always be part of the pilot, so adjustments can be made as needed before full implementation. Step 9: Provide accommodations as needed (see Section 5). Accommodations are addi- tional supports that students may need when taking assessments or completing schoolwork. When assessments are designed accessibly, the need for accommodations may be reduced but will not be eliminated. Accommodations should be provided to students as needed. Decisions about accommodations should be made by the educational team supporting the student, including the student himself/herself, a family member, the teacher, and other support service providers. It is best to document these supports so that they are consistently used in both instruction and assessment. If the student has an IEP or an academic support plan, these accommodations should be documented in these plans. Decisions on the type of accommodations provided should be based on the student’s particular needs and not on the disability category or other characteristics. Never assume a student’s needs by defining them by the student’s disability status, gender, ethnicity, or other characteristics. Once accommodations are used, they should be reviewed to see how they supported the student and if changes might be needed in future. Section 5 of this toolkit provides additional information on accommodations as well as information on the differences between accommodations and modifications. Step 10: Administer with a universal design approach (see Section 3). This step builds on CAST’s UDL guidelines. These guidelines suggest that educational experiences should provide (1) multiple means of engagement, (2) multiple means of representation, and (3) multiple means of action and expression. Translated into the assessment context, this would mean that UDA will • Allow students to access information and items both visually and auditorily; • Allow students to respond either verbally or through pointing/circling/writing; Using Principles of UDA to Design 59 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA • Ensure assessors are engaging and warm so that students can easily converse with them. If possible, students should already know the assessors; and • Adhere to domain-specific recommendations found in Step 3 above. Step 11: Use data analysis strategies. Once the UDA strategies have been implemented and UDA is administered, further inferences about its accessibility can be made through data analysis. Ideally, the data would be analyzed by statisticians or other professionals who can select the best method to review the data. In addition, when presenting data, descriptive statistics and graphical representations should be used that can help make the data accessi- ble to a diverse group of stakeholders. Step 12: Share results with key stakeholders. Although UDA is a global approach aimed at improving assessment accessibility, the interpretation and use of data should be highly contextualized. Assessment data need to be interpreted within the framework of the cultural context and educational structure of where the assessment took place. Debriefing meetings with national and local stakeholders can help answer questions about why certain findings may have arisen and what to do with emergent results in relation to assessment or adminis- tration revisions. Ideally, messages should be tailored to address how the results apply to the specific context of different stakeholders. Step 13: Learn and continuously improve UDA tools and processes. The pursuit for inclu- sion must be relentless and ongoing. After every UDA administration, lessons will be learned from the process of administration and the test data. There is no endpoint to a universally designed assessment. Rather, shared expertise of stakeholders and assessment data allow a version to be developed, piloted, and eventually fully administered. After the pilot and comprehensive administrations, data should be analyzed to test accessibility hypotheses. Additionally, as accessibility features are built into assessments, student response data can be examined to determine the impacts of features. Further, qualitative interviews with stu- dents can also inform the process. In sum, a test is never ‘universally designed’. Rather, UDA strategies can be used to improve assessment accessibility and accessibility should always be reviewed and, if necessary, revised in subsequent versions. Using Principles of UDA to Design 60 Toolkit Accessible Learning Assessments Universally Designed EGRA and EGMA Photo Credit: World Bank 9. Using Assessment Data This section reviews... • How to use assessment data in LMICs Assessment data can be useful to teachers • Measure individual student progress and school systems in a variety of ways. toward specific benchmarks, and Assessments can: • Ensure that all students benefit from • Inform teachers on how students have instruction (Spinelli 2012). responded to instruction and on how to adjust teaching methods as needed, Assessments also provide information to teachers about students who may struggle • Measure the individual student’s in the classroom and who may be at risk of achievement compared to grade-level academic failure. Students may struggle for peers, many reasons, such as being in a language Using Principles of UDA to Design 61 Toolkit Accessible Learning Assessments Using Assessment Data minority, being exposed to trauma, expe- FIGURE 3: Three Tiers for Instruction and riencing poverty and hunger, or having Support a disability. Using assessment data to identify struggling learners is preferred as other methods of identification may be influenced by teacher biases (Alahmari 2019). For example, many checklists and evaluation tools ask teachers to provide their opinions of students. These opinions can lead to the determination of disability based on gender, behavior, or other demo- graphics, such as race or religion. Using assessment data to determine which students may benefit from additional intervention and support is a core element of a multi-tiered system of support (MTSS). MTSS is a schoolwide method for providing students with instruction and behavioral support at three levels: whole classroom instruction, small group instruction, and • Regular screening and progress intensive/individualized instruction. monitoring Essentially, MTSS allows teachers to adjust the intensity of instruction based • Data-based decision-making by the on students’ assessment data as well as instructional team.” (Hayes, Turnbull, provide class wide and behavioral supports and Moran 2018) as needed (Hayes, Turnbull, and Moran 2018). The key features of MTSS include the Figure 3 provides an overview of the three following: tiers for instruction and support. In Tier 1, students are provided with high-quality, • “ The creation of an MTSS team for evidence-based instruction by a trained planning and implementation teacher and then using formative and summative assessments indicates which • Three tiers of increasing intensive students may not be meeting academic instruction milestones compared to their peers. Based on these assessments, in Tier 2, students • Use of evidence-based instructional are provided additional targeted interven- practices tion within a small group setting in addition to the general curriculum over a period. • Differentiated instruction through UDL Students who continue to demonstrate little Using Principles of UDA to Design 62 Toolkit Accessible Learning Assessments Using Assessment Data progress through formative and summative 2009). Other aspects of MTSS that would assessments may benefit from receiving apply to LMICs include individualized supports in Tier 3. These supports can include accommodations and • Focusing on the needs of all students in adaptions in assessments and in the class- the classroom including those who may room setting as well as special education struggle to learn, services within the inclusive classroom setting (RTI Action Network 2022). Table 9 • Providing different levels of instruction provides additional information on inclusive based on need, practices that could be used in a classroom with the MTSS model. • Establishing a collaborative culture where there is a collective responsibility One of the primary benefits of using to teach all students, and assessment data to provide students with a tiered level of support is that students who • Recognizing that the tiers are fluid are struggling can be proactively identified and students may move between tiers before they fail (Reynolds and Shaywitz based on their needs and progress. TABLE 9: Suggestions for Inclusive Practices by Tier Tier Recommended Inclusive Practices Tier 1 • Use UDL practices for all students. • Use UDA for assessment. Tier 2 • Provide small group instruction. • Provide accommodations if needed; for example, a child who is easily distracted in large groups could benefit from taking exams in a quiet setting. Tier 3 • Provide individualized instruction and evaluate learners for possible disability. • Provide accommodations and explore if modifications are needed. For example, during a formative assessment, students with an identified intellectual disability may take a spelling test with fewer questions and more aligned with their learning level. Using Principles of UDA to Design 63 Toolkit Accessible Learning Assessments Using Assessment Data MTSS should be used to facilitate inclusion and never be used to remove students from the classroom. It is important to recognize that while the general principles of MTSS are applicable to LMICs, other aspects of this approach must be further researched and piloted within each respective country to ensure that interventions reflect the country’s context. For example, when MTSS is applied in HICs, it is assumed that approximately 80 percent of students will be able to make progress with quality evidence-based curriculum and instruction (Shapiro 2014). However, this percentage may differ in countries with high incidences of trauma, poverty, or stunting, all of which signify that more students may struggle to learn. Thus, the percentage of students succeeding with only Tier 1 interventions may be lower in other countries. The following are suggested steps on how MTSS could be applied in LMICs. • Step 1: Ensure that all students in early grades receive vision and hearing screen- ings. Many LMICs have yet to administer routine and universal vision and hearing screenings within the classroom setting. Some level of vision and hearing screenings must be administered to rule out if learning challenges may be related to not seeing the content or not hearing the instruction. These vision and hearing screenings should engage caregivers, and schools should gain caregivers’ consent before screening students. Once completed, schools should refer students to medical clinics, where available, to receive additional testing and potentially assistive devices, such as glasses. Teachers can also modify the classroom environment to support students with known vision and hearing challenges (Hayes, Turnbull, and Moran 2018). • Step 2: Use formative and summative assessments using principles of UDL. Based on the guidance provided in this toolkit and in other evidence-based materials, all assessments should be designed to be accessible and to utilize the principles of UDL. Teachers may want to use data from curriculum-based assessments to support students in the classroom with tiered interventions (that is, assessments that use observations of students’ performance on the local curriculum; Deno 1987) instead of data from EGMA, EGRA, or other standardized tests. Teachers can then consistently monitor progress through assessments based on relevant curricular content. • Step 3: Establish benchmarks and procedures for how teachers recognize and define success. The benchmarks and procedures for determining a student’s success may vary from country to country, within regions, and even between schools. Many countries’ benchmarks related to literacy and numeracy already exist. If so, these can be used or adapted by a particular school or classroom. Using Principles of UDA to Design 64 Toolkit Accessible Learning Assessments Using Assessment Data • Step 4: Determine where a student scores according to benchmarks. Once data are obtained, teachers, with school leadership, should review student assessment scores and identify which scores may indicate a student’s progress at an acceptable rate and which scores may indicate progress that is lower than the determined benchmark, placing the student at risk of poor learning or poor social, emotional, and behavioral outcomes (Hoover 2011). For example, a school has a grade-level or school-determined benchmark that students should read at 100 words per minute. However, a particular student is consistently reading at 70 words a minute, despite the evidence that many of their classroom peers have obtained the benchmark. The lower score for reading words per minute may indicate that this student would benefit from additional support in tiered instruction. Conversely, if the majority of the students in the class are not meeting this determined benchmark for reading words per minute, the selected benchmark may be too high and should be reexamined. • Step 5: Provide small group instruction to students below the identified benchmark. Students who continue to struggle over time compared to their peers often benefit from small group instruction (Shapiro 2014). During this time, all students may be placed in small groups based on their learning strengths and needs. For example, students can be placed together in groups with other students who have similar challenges. One group may focus on decoding, another group may focus on fluency or comprehension, and so on. During small group instruction, students in the class who are doing better in literacy (or math) could use this time to work on more complex tasks. Suggestions for small group instruction include the following: | Organize a collaborative schoolwide team of teachers and staff to address the learn- ing, behavioral, and social-emotional needs of all students. | Engage with school leadership, the schoolwide team, caregivers, and community members to find solutions for supporting small group instruction. | Strategically use school volunteers, such as caregivers and community members, to allow small group instruction and support other students in the classroom during small group instruction time. | Arrange groups of six or fewer students to be led by a teacher or a school volunteer. Teachers should lead groups of students with the most challenging learning and behavioral needs. | Strategically place students together based on similar learning needs. | Allow time for each teacher-volunteer/student group to meet three to five times per week. Using Principles of UDA to Design 65 Toolkit Accessible Learning Assessments Using Assessment Data | Allow 30 minutes for each teacher-volunteer/student session. | Allow a time and place within the school day instead of outside school hours to reduce the burden on teachers, volunteers, and students. • Step 6: Continually monitor students’ progress. Progress monitoring is an important element of MTSS as it can help teachers understand how small group instruction helps each student learn. Although progress should be recorded for all students, particular focus should be given to those students who are unable to meet the determined benchmarks and are placed in small group instruction. Students who are successfully learning through teacher-led small group instruction may be able to move toward fewer days per week in small group instruction or to instruction led by a volunteer. However, for students who do not continue to make progress, additional time and supports may be needed. Again, students may struggle for various reasons including short-term chal- lenges, such as a disruption to their home life or an illness that may subside over time. In these cases, dips and plateaus in performance should be anticipated and considered when planning small group instruction. • Step 7: Provide individualization and accommodations to students who continue to struggle to learn. Students who continue to struggle with teacher-led small group instruction may benefit from additional supports including individualization and accommodations (see Section 5). All students have the right to learn in an inclusive environment so the need for individualization does not signify that a student should be removed from the classroom and placed in an alternative setting, such as a resource room or segregated school. Teaching at the Right Level Teaching at the right level (TaRL) is a remedial education program developed by the Pratham Education Foundation in India and has since been used in several countries in Africa (Banerjee et al. 2016). It supports the premise that students have the ability to learn at their own pace and allows teachers to provide additional support to students as needed. Although TaRL and MTSS are slightly different, both recognize that stu- dents may require various levels of instruction and this determination should be based on assessment data and not teacher opinion. Using Principles of UDA to Design 66 Toolkit Accessible Learning Assessments Using Assessment Data • Step 8: Train teachers. For MTSS to be successful, the school must reflect a unique cultural context that values the success of all students and recognizes the benefits of inclusive education. Importantly, teachers will require sufficient training on use of assessment data and on procedures for monitoring students’ performance. Teacher training should include the following: | Disability awareness so that teachers understand the benefits of inclusive education and that MTSS and assessment data are used to support inclusion and do not sig- nify a justification for segregation. | How to collect, analyze, and use assessment data to support decision-making when developing supports for students. | How to develop a collaborative whole-school environment to support MTSS. Using Principles of UDA to Design 67 Toolkit Accessible Learning Assessments Using Assessment Data 10. Conclusion As outlined within this UDA toolkit, incorporating assess- ment flexibility (without changing constructs) gives students with disabilities ample opportunities for inclu- sion. Cases will arise, however, in which students will need additional accommodations. The two-strategy approach of first applying accessibility principles (informed by UDA) and then accommodations for students will allow greater participation of students who are blind or have low vision, are deaf or hard of hearing, have intellectual disability, or have physical disabilities. As UDL and UDA build on each other, UDA is also more likely to be successful if UDL is being used consistently within the classroom. By committing to implement both UDL and UDA, LMICs can give all students the opportu- nity to both learn and show what they have learned. As a result, learners who are comfortable expressing what they learn in different ways within classroom instruction will be more comfortable doing so in assessments as well. By committing to both UDL and UDA, more accurate informa- tion can be gathered on learning within the classroom—for all students. Using Principles of UDA to Design 68 Toolkit Accessible Learning Assessments Conclusion References Abedi, J. 2021. Accommodations and Universal Design. In The Routledge Handbook of Language Testing, edited by G. Fulcher and L. Harding, 306–321. Routledge. Alahmari, A. 2019. “A Review and Synthesis of the Response to Intervention (RtI) Literature: Teachers’ Implementations and Perceptions.” International Journal of Special Education 33 (4): 894–909. (AERA) American Educational Research Association, (APA) American Psychological Association, and (NCME) National Council on Measurement in Education. 2014. Standards for Educational and Psychological Testing. AERA.  Andrade, H., and G. J. Cizek, eds. 2010. Handbook of Formative Assessment. Routledge. Banerjee, A., R. Banerji, J. Berry, E. Duflo, H. Kannan, S. Mukherji, ... and M. Walton. 2016. “Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of ‘Teaching at the Right Level’ in India.” No. w22746), National Bureau of Economic Research. Bolt, S. E., and M. Thurlow. 2004. “Five of the Most Frequently Allowed Testing Accommodations in State Policy: Synthesis of Research.” Remedial and Special Education 25 (3): 141–152. https://doi.org/10.1177/07419325040250030201. Bouck, E. C., and H. Long. 2021. “Assistive Technology for Students with Disabilities: An Updated Snapshot.” Journal of Special Education Technology 36 (4): 249–257. Buzick, H. M., and C. C. Laitusis. 2010. “Using Growth for Accountability.” Educational Researcher 39 (7): 537–544. https://doi.org/10.3102/0013189x10383560. CAST (Center for Accessible Special Technologies). 2018. Universal Design for Learning Guidelines Version 2.2. http://udlguidelines.cast.org. Cawthon, S., and R. Leppo. 2013. “Assessment Accommodations on Tests of Academic Achievement for Students Who Are Deaf or Hard of Hearing: A Qualitative Meta-Analysis of the Research Literature.” American Annals of the Deaf 153 (2). Gallaudet University Press. Using Principles of UDA to Design 69 Toolkit Accessible Learning Assessments References Center for Universal Design, North Carolina State University. 1997. The Principles of Universal Design. https://projects.ncsu.edu/ncsu/design/cud/about_ud/udprinciplestext.htm. Clarke, M. 2012. “What Matters Most for Student Assessment Systems: A Framework Paper.” Systems Approach for Better Education Results (SABER) Student Assessment Working Paper No. 1, World Bank, Washington, DC. https://openknowledge.worldbank.org/ handle/10986/17471. Clarke, M., and D. Luna-Bazaldua. 2021. Primer on Large-Scale Assessments of Educational Achievement. World Bank Publications. Clarke. M, Diegoando. A and Luna Bazaldua, D. A., 2022. Planning a Large-Scale Assessment of Student Achievement: What Are Essential Elements of the Assessment Framework? National Assessments of Educational Achievement Guidance Note. World Bank. Cortiella, C. 2005. No Child Left Behind: Determining Appropriate Assessment Accommodations for Students with Disabilities. National Center for Learning Disabilities. https://www.readingrockets.org/article/no-child-left-behind-determining-appropri- ate-assessment-accommodations-students-disabilities. Davies, M. 2018. “International Policies That Support Inclusive Assessment.” In Handbook of Accessible Instruction and Testing Practices, edited by Elliott S. N., Kettler R. J., Beddow P. A., and Kurz A., 37–58. Springer, Cham. Dembitzer, L., and R. J. Kettler. 2018. “Testing Adaptations: Research to Guide Practice.” In Handbook of Accessible Instruction and Testing Practices, edited by Elliott S. N., Kettler R. J., Beddow P. A., and Kurz A., 213–230. Springer, Cham. Deno, S. L. 1987. “Curriculum-Based Measurement.” Teaching Exceptional Children 20 (1): 40–42. Downing, J. E. 2005. Teaching Literacy to Students with Significant Disabilities: Strategies for the K-12 Inclusive Classroom. Thousand Oaks, CA: Corwin Press, Sage Publications Company. Erickson, R., and M. Thurlow. 1996. 1995 State Special Education Outcomes-Longitudinal Trends in How States are Assessing Educational Outcomes for Students with Disabilities (NCEO State Survey Report). Using Principles of UDA to Design 70 Toolkit Accessible Learning Assessments References Fleisher, C. 2022. Money Problems: Do Monetary Themed Question on Standardized Tests Have a Negative Impact on Students of Low Socioeconomic Status? American Economic Association. https://www.aeaweb.org/research/ attention-capture-monetary-standardized-tests. Haladyna, T. M., and S. M. Downing. 2004. “Construct-Irrelevant Variance in High-Stakes Testing.” Educational Measurement: Issues and Practice 23 (1): 17–27. Hamilton, K., and E. Kessler. n.d. Accommodations and Modifications: Wait, They’re Not the Same?https://www.gadoe.org/CurriculumInstructionandAssessment/ SpecialEducationServices/Documents/CoTeaching%20Modules/Module%204/27%20 nichcy.org-Accommodations_and_modifications_Wait_theyre_not_the_same.pdf . Hanna, E. I. 2005. Inclusive Design for Maximum Accessibility: A Practical Approach to Universal Design. PEM Research Report. Hayes, A., A. Turnbull, and N. Moran. 2018. Universal Design for Learning to Help All Children Read: Promoting Literacy for Learners with Disabilities. USAID. Hoover, J. J. 2011. “Making Informed Instructional Adjustments in RTI models: Essentials for Practitioners.” Intervention in School and Clinic 47 (2): 82–90. Johnstone, A. 2003. Effective Practice in Objective Assessment. Hull, UK: LTSN Physical Sciences Centre. Johnstone, C., J. Higgins, and G. Fedorchak. 2019. “Assessment in an Era of Accessibility: Evaluating Rules for Scripting Audio Representation of Test Items.” British Journal of Educational Technology 50 (2): 806–818. Katz, J. 2013. “The Three Block Model of Universal Design for Learning (UDL): Engaging Students in Inclusive Education.” Canadian Journal of Education 36 (1): 153–194. Kettler, R. J. 2015. “Adaptations and Access to Assessment of Common Core Content.” Review of Research in Education 39 (1): 295–330. Koenig, A. J., and M. C. Holbrook. 1995. Learning Media Assessment of Students with Visual Impairments: A Resource Guide for Teachers. Texas School for the Blind and Visually Impaired, Business Office, 1100 West 45th St., Austin, TX 78756-3494. Using Principles of UDA to Design 71 Toolkit Accessible Learning Assessments References Landau, J. K., J. R. Vohs, and C. A. Romano. n.d. All Kids Count. Parents Engaged in Educational Reform (PEER Project), Federation for Children with Special Needs, Boston, Massachussetts. Lazarus, S. S., C. J. Johnstone, K. K. Liu, M. L. Thurlow, A. R. Hinkle, K. Burden, and A. R. Hinkle. 2022. An Updated State Guide to Universally Designed Assessments. Lazarus, S. S., S. J. Thompson, and M. L. Thurlow. 2006. How Students Access Accommodations in Assessment and Instruction: Results of a Survey of Special Education Teachers. EPRRI Issue Brief Seven. Educational Policy Reform Research Institute. Linn, R. L. 2001. “A Century of Standardized Testing: Controversies and Pendulum Swings.” Educational Assessment 7 (1): 29–38. Martone, A., and S. G. Sireci. 2009. “Evaluating Alignment between Curriculum, Assessment, and Instruction.” Review of Educational Research 79 (4): 1332–1361. Maryanti, R. 2021. “Assessment of Mathematical Abilities of Students with Intellectual Disabilities during the COVID-19 Pandemic.” Indonesian Journal of Community and Special Needs Education 1 (2): 47–52. Meyer, A., D. H. Rose, and D. Gordon. 2014. Universal Design for Learning: Theory and Practice. Wakefield, MA: CAST Professional Publishing. National Center for Learning Disabilities. n.d. Understanding the Standards-based Individualized Education Program (IEP) https://www.advocacyinstitute.org/resources/ UnderstandingStandards-basedIEPs.pdf Newton, P. E. 2007. “Clarifying the Purposes of Educational Assessment.” Assessment in Education 14 (2): 149–170. Piper, B, Bulat, J, Kwayumba, D, Oketch, J and Gangla, L. (2019). Measuring literacy outcomes for the blind and for the deaf: Nationally representative results from Kenya. International Journal of Educational Development. 69. 1-8. 10.1016/j.ijedudev.2019.05.002. Perkins School for the Blind. n.d. Math Test Accommodations for Low Vision. https://www. perkins.org/resource/math-test-accommodations-low-vision/. Using Principles of UDA to Design 72 Toolkit Accessible Learning Assessments References Region 4 Service Center. 2015. Assessment of Braille Literacy Skills (ABLS): UEB and EBAE. Houston TX: Region IV Education Service Center. Rello, L., and R. Baeza-Yates. 2013. “Good Fonts for Dyslexia.” ASSETs ’13: Proceedings of the 15th International ACM SIGACESS Conference on Computers and Accessibility. Article 14, page 1–8. Reynolds, C. R., and S. E. Shaywitz. 2009. “Response to Intervention: Ready or Not? Or, from Wait-to-Fail to Watch-Them-Fail.” School Psychology Quarterly 24 (2): 130. Rose, S., L. Barkmeier, S. Landrud, and V. Hoekstra. 2008. Assessment of Students Who Are Deaf and Hard of Hearing. Minnesota Center for the Deaf and Hard of Hearing. Accessed November 19, 2021. https://www.cehd.umn.edu/dhh-resources/assessment/ AssessmentManual.pdf. Response to Intervention (RTI) Action Network. 2022. What Is RTI? http://www.rtinetwork. org/learn/what/whatisrti. RTI (Research Triangle Institute) International. 2015. Early Grade Reading Assessment (EGRA) Toolkit, Second Edition. Washington, DC: USAID.  RTI International. 2022. Science of Learning: Assessment-Informed Instruction. https://www. rti.org/publication/assessment-informed-instruction Salend, S. J. 2009. “Technology-Based Classroom Assessments: Alternatives to Testing.” Teaching Exceptional Children 41 (6): 48–58. Salvador, K., and M. E. Culp. 2022. “Intersections in Music Education: Implications of Universal Design for Learning, Culturally Responsive Education, and Trauma-Informed Education for P–12 praxis.” Music Educators Journal 108 (3): 19–29. Shapiro, E. 2014. Tiered Instruction and Intervention in a Response-to-Intervention Model. Center for Promoting Research to Practice, Lehigh University, Bethlehem, PA. http://www.rtinetwork.org/essential/tieredinstruction/ tiered-instruction-and-intervention-rti-model. Spinelli, C. 2012. Classroom assessment for students in special and general education. Upper Saddle River, NJ: Pearson Education. Using Principles of UDA to Design 73 Toolkit Accessible Learning Assessments References Stone, E., L. Cook, C. C. Laitusis, and F. Cline. 2010. “Using Differential Item Functioning to Investigate the Impact of Testing Accommodations on an English-Language Arts Assessment for Students Who Are Blind or Visually Impaired.” Applied Measurement in Education 23 (2): 132–152. https://doi.org/10.1080/08957341003673773. Thompson, S., and M. Thurlow. 2002. Universally Designed Assessments: Better Tests for Everyone! NCEO Policy Directions. Trent, S. D. and M. B. Truan. 1997. Speed, Accuracy, and Comprehension of Adolescent Braille Readers in a Specialized School. https://doi.org/10.1177/0145482X9709100509. United Nations Disability Inclusion Strategy. 2021. Guidelines on Consulting Persons with Disabilities: Indicator 5. https://www.un.org/sites/un2.un.org/files/un_disability-inclu- sive_consultation_guidelines.pdf. World Bank. 2021. Guidance Note: Criteria for the World Bank’s Disability-Inclusive Investment Project Financing (IPF) in Education. Washington, DC. https://documents1. worldbank.org/curated/en/712711627321467972/pdf/Criteria-for-the-World-Bank-s- Disability-Inclusive-Investment-Project-Financing-IPF-in-Education-Guidance-Note.pdf. World Bank, Inclusion International and Leonard Cheshire. (2019.) Every learner matters: Unpacking the learning crisis for children with disabilities. Washington, DC. https:// www.leonardcheshire.org/sites/default/files/2019-06/Every%20Learner%20Matters%20 FINAL%20DIGITAL.pdf World Blind Union. 2020. “WBU Statement for World Braille Day – 4 January 2021.” https:// worldblindunion.org/blog/news/wbu-statement-for-world-braille-day-4-january-2021/. Using Principles of UDA to Design 74 Toolkit Accessible Learning Assessments References Glossary Accommodation: Changes that reduce Braille: A widely used touch system of barriers and allow students to effectively reading and writing for people who are receive information and express learning. blind that uses a special code made from These can include changes in the presenta- six raised dots on a grid (World Blind Union tion, response, timing, scheduling, setting, 2021). or language used. Construct: The concept or content being Adaptions: All students engage with the tested. same content or construct, but the timing or the way items are presented or the way Disability: A social concept that resides not students respond to assessment items may in a person’s specific impairment but in the be adapted (Bolt and Thurlow 2004), for interactions between the impairment and example, adapting a test normally delivered the attitudinal and environmental barriers in text format to braille format or allowing a they face (World Bank 2021). student to respond in sign language instead of speech. Formative assessment: Provides real-time information that teachers can use to guide Assessment: The process of gathering and day-today instructions or to tailor teaching evaluating information on what students to the needs of individual students (Clarke know, understand, and can do (Clarke and Luna-Bazaldua 2021). 2012). Early Grade Math Assessment (EGMA): Augmentative and alternative com- A test developed by RTI International to munication (AAC): A set of tools and measure how students in the same coun- strategies that an individual uses in com- tries are acquiring mathematic skills. munication. These can include speech, text, gestures, facial expressions, touch, Early Grade Reading Assessment sign language, symbols, pictures, and (EGRA): A test developed by RTI speech-generating devices (International International to measure how well primary Society for Augmentative and Alternative grade students in LMICs are acquiring Communication)11. reading skills primarily used by early grade reading projects. 11 International Society for Augmentative and Alternative Communication. What Is AAC? https://isaac-online.org/english/ what-is-aac/. Using Principles of UDA to Design 75 Toolkit Accessible Learning Assessments Glossary Large-scale assessment: Provides infor- Summative assessment: Provides infor- mation on the aggregated performance mation on what students have learned that levels among students in the education can guide decisions on the progression system for a particular curriculum area and of students through the education sys- at a particular age or grade level (Clarke tem or decisions on education system and Luna-Bazaldua 2021). policy (Clarke and Luna-Bazaldua 2021). Administered after the teacher completes Modifications: Changes to what a student instruction (AERA, APA, and NCME 2014). is assessed on or what the student is expected to learn. Universal design for assessment (UDA): Assessments that are designed to allow the Organization of persons with disabilities equitable participation of the widest range (OPD): An organization led, directed, and of students (Thompson and Thurlow 2002). governed by persons with disabilities who compose a clear majority of its member- Universal design for learning (UDL): ship (United Nations Disability Inclusion Adaptations to make learning more accessi- Strategy 2021). ble to persons with disabilities that may be helpful to all individuals. Using Principles of UDA to Design 76 Toolkit Accessible Learning Assessments Glossary Annex A: Images of Number Lines Used in Tajikistan UDA Pilot Image Description: Number line used for Grade 2 UDA test. The numbers 8 and 4 are shown for students, and a number line from 0 to 15 is provided. Using Principles of UDA to Design 77 Toolkit Accessible Learning Assessments Annex A: Images of Number Lines Used in Tajikistan UDA Pilot Image Description: Number line used for Grade 4 UDA test. The numbers 67298 and 97300 are shown for students, and a number line from 65000 to 100000 is provided. Using Principles of UDA to Design 78 Toolkit Accessible Learning Assessments Annex A: Images of Number Lines Used in Tajikistan UDA Pilot Annex B: Word Problem Images Used in Tajikistan UDA Pilot Image Description: Three sets of pictures are present in the image. The first picture has students in a classroom, with five boys and girls in the top picture, but two leave in the bottom picture. The second group of pictures shows five football players on a bench, and in the second picture two players leave the bench. Finally, the third set of pictures shows a bowl with five apples, and in the second picture two apple cores are present, indicating two apples had been eaten. Using Principles of UDA to Design 79 Toolkit Accessible Learning Assessments Annex B: Word Problem Images Used in Tajikistan UDA Pilot Annex C: Accessibility Checklist Accessibility Guideline Yes/No Recommended Changes Paper-Based Assessments Are instructions presented in a language the student understands? (Adapted from Thompson et al. 2002) Is an example of how to complete a problem provided for every new item set or domain? (Adapted from Thompson et al. 2002) Are common words in the language of student instruction used for assessment instructions? (Adapted from Thompson et al. 2002) Are technical terms avoided unless necessary for the construct of the assessed item? (Adapted from Thompson et al. 2002) Is the item free from unnecessary language requirements that are unrelated to the construct assessed? (Adapted from Thompson et al. 2002) Do illustrations support the item’s comprehension? (Adapted from Thompson et al. 2002) Is the text in at least 14-point font? (Adapted from Thompson et al. 2002) Is there adequate white space on pages and between images, numbers, and so on to facilitate the comprehension of items? (Adapted from Thompson et al. 2002) Computer-Based Assessments Is the text in 18-point font? (Section 508 guidelines) Using Principles of UDA to Design 80 Toolkit Accessible Learning Assessments Annex C: Accessibility Checklist Accessibility Guideline Yes/No Recommended Changes Can the test be resized as needed on the platform? (Section 508 guidelines) Is the information not reliant on color alone? (Section 508 guidelines) Can instructions be accessed verbally and textually? (Section 508 guidelines, UDL guidelines) Can items be accessed verbally and textually? (Section 508 guidelines, UDL guidelines) Is alternative text (alt text) provided for all images? (Section 508 guidelines, UDL guidelines) Are there keyboard equivalents for every mouse requirement? (Section 508 guidelines) Using Principles of UDA to Design 81 Toolkit Accessible Learning Assessments Annex C: Accessibility Checklist Annex D: Additional Accommodations by Disability Type This annex provides additional accommodations that may support students with a variety of disabilities. Additional Accessibility Features for Students Who Are Blind or Have Low Vision or Are Deaf or Hard of Hearing Students who are blind or have low vision and students who are deaf or hard of hearing often require additional accessibility features. Students who are blind or have low vision may require a variety of alternate formats, accessibility features, and accommodations to have full access to assessments. No single method will work for every student. Some students require more than one accommodation and may use alternate formats and accommodations interchangeably. Additionally, the student needs to be familiar with the accessibility features and use them on a regular basis. Table D.1 provides suggestions for accommodations for students who are blind or have low vison. TABLE D.1: Suggested Accessibility Options for Blind or Low Vision Students Alternate Formats Print all assessment sheets on braille paper or cardstock and reuse each assessment no more than three times; after each use, ensure Braille all dots are still raised. Ensure that one line of spacing is between all lines of braille to avoid students ‘catching’ the dots from the line above. Large print Use 18 points or larger to allow for materials to be within large print. Using Principles of UDA to Design 82 Toolkit Accessible Learning Assessments Annex D: Additional Accommodations by Disability Type Audio version of text including description of images and other Audio versions graphics can help ensure accessibility of information. Use screen readers, text-to-speech software, and keyboard access Digital versions through braille or switches. Computer-generated testing can be compatible with braille displays and screen-magnification software. Built-In Accessibility Features Maps, diagrams, and other technical illustrations may require the use of tactile graphics, and developing tactile graphics requires specialized knowledge and skill. Many students who use tactile graphics also require a written description of the graphic to Tactile graphics accompany the tactile version. Haptics communication, such as using touch to draw images on the student’s body, is also a strategy a teacher or assessor can use to provide a student visual information about images. Test items should not rely solely on pictures for information needed to answer the test items. Include image descriptions of all non-text items Image descriptions to provide more detailed descriptions of an image. Alternative text or ‘alt text’ provides a shorter, more basic description of an image. Accommodations Students who are blind or have low vision or are deaf or hard of Devices hearing should be allowed devices, including magnifiers, screen readers, recorders, and closed-captioned televisions (CCTVs). An individual who can read instructions, including test instructions Human readers or other elements of written text provided in the classroom setting. Students who are blind or have low vision may experience eye Additional time and fatigue, and braille reading consumes more time than print reading breaks (Trent and Truan 1997). Using Principles of UDA to Design 83 Toolkit Accessible Learning Assessments Annex D: Additional Accommodations by Disability Type Scribes Students are allowed to dictate responses to a scribe or notetaker. An accessible calculator should be permitted if scientific or Calculators nonscientific calculators are allowed for the general student population. Rulers, protractors, and other devices help students keep their Manipulatives place on the test and draw straight lines. An abacus is often used as an alternative to pen and paper to work through math problems. Students are permitted to respond to test questions in an alternate Alternate response format, such as orally, via a braillewriter, with a slate and stylus, and through word processing programs. Bold, dark, writing Students are allowed to use markers such as Sharpie pens. devices Some students may require a light on their desk to illuminate the Task lighting test paper. Assessments should use single-sided, bold-line, or raised-line Specialized paper paper. Some students who have low vision require specific color contrast Color contrast between the background and print color. White-colored paper often creates a glare that interferes with visual access. Using Principles of UDA to Design 84 Toolkit Accessible Learning Assessments Annex D: Additional Accommodations by Disability Type Accessibility Features for Students Who Are Deaf or Hard of Hearing Students who are deaf or hard of hearing constitute a diverse group with a wide range of language and cultural backgrounds. Some students who are deaf or hard of hearing use sign language while others use hearing aids, cochlear implants, or other assistive devices to maximize residual hearing. Many students use a combination of these devices. Table D.2 provides accommodations that may be useful for students who are deaf and hard of hearing. TABLE D.2: Suggested Accommodations for Deaf and Hard of Hearing Students Accommodations Sign language interpreters should be skilled in test situations and Sign language be familiar with the content being tested. If possible, students interpreters should use the same interpreters for testing and instruction. Extended time on Additional time for tests or having the option of tests that are not tests and interval timed as well as frequent breaks, depending on the needs of the breaks students Captions should be used for any videos or instructions. This can be Captioned media in combination with a sign language interpreter. A scribe will record the responses that a student gives in sign Scribes language. Assistive listening Students are allowed to use devices such as frequency modulation devices (FM) systems, loop systems, and pocket talkers. Students have access to transcripts for audio/video portions of the Transcripts test. Using Principles of UDA to Design 85 Toolkit Accessible Learning Assessments Annex D: Additional Accommodations by Disability Type Students with Multiple/Complex Disabilities and Students Who Are Deafblind With a global commitment to the inclusion of all students with disabilities in accountability efforts, it is imperative to consider the needs of students with multiple and complex disabil- ities including students who are deafblind. Accommodations and modifications allow many students with disabilities to participate in a variety of assessments. For students who have multiple/complex disabilities and for those who are deafblind, participation in assessments may require a more thoughtful, individualized approach, and in some cases, an alternative assessment may be an appropriate option. Educators need to understand that students with multiple/complex disabilities and those who are deafblind often have such significant challenges with access to information, com- munication, and learning that they are perceived to have an intellectual disability. If students have not learned formal communication and language systems and do not have access to incidental learning opportunities, they do not have the ability to learn new concepts and express their thoughts and ideas. However, this should not reflect on their intellectual capa- bilities. Having access to an educational program that has expertise in educating students with complex needs is imperative for a student to learn and thrive. It is critical that the sup- port and accommodations that are part of the student’s instructional program are present in the testing environment. Given the heterogeneity of students who have multiple/complex disabilities and those who are deafblind, students may need to be supported with a variety of accommodations and modifications to meet their specific needs. For example, • A student who is deafblind without enough residual vision and hearing to access visual sign language or print may require a tactile sign language interpreter and a braille ver- sion of the test. • A student with limited motor ability who cannot use physical movement to indicate an answer and who is also deaf may need a sign language interpreter and alternative means of indicating an answer (such as using eye gaze instead of a written answer). As appropriate solutions for these students can be complex, teachers and other support staff who work with these students on a regular basis should be involved in the planning and implementation of assessments to ensure maximum access. In addition to the accommoda- tions listed for students who are blind or have low vision or who are deaf or hard of hearing, Table D.3 provides suggestions for accommodations that can be considered for students with multiple/complex disabilities and those who are deafblind. Using Principles of UDA to Design 86 Toolkit Accessible Learning Assessments Annex D: Additional Accommodations by Disability Type TABLE D.3: Suggested Accommodations for Deafblind Students and Students with Multiple Disabilities Accommodations Proper positioning with This may require a tilt screen or a raised desk. Students should be full access to well supported so that they can focus on the assessment and not on usable vision/ trying to balance and support their body. hearing Extended time on Additional time for tests or having the option of tests that are not tests and interval timed as well as frequent breaks depending on the needs of the breaks students. Students should be allowed and/or supported to use their preferred Communication mode of communication, including picture symbols, AAC devices, and preferences object symbols. Students should have access to augmentative communication devices AAC such as an iPad or other technology devices that they regularly use to communicate. Give the assessment during periods when the student is in an alert state. Some students, due to medical conditions or medicines Timing that they take, have times during the day when they perform more optimally. Multiple means of Students may use both pointing and eye gaze to indicate a test indicating choice answer depending on their physical status that day. Students should have a workspace that is large enough to hold their Large workspace AAC devices, manipulatives, and other support devices. Using Principles of UDA to Design 87 Toolkit Accessible Learning Assessments Annex D: Additional Accommodations by Disability Type Accommodations Students should have time to orient to the test environment if they are Environment unfamiliar with it. There is contrast between the background and print, pictures, and Color contrast objects. Other Students who are deafblind have access to tactile interpreting, close- accommodations vision interpreting, or other modifications. Using Principles of UDA to Design 88 Toolkit Accessible Learning Assessments Annex D: Additional Accommodations by Disability Type www.inclusive-education-initiative.org