Accessibility in User-Centered Design: Preparing for Usability Testing

The Evaluating for Accessibility page provides guidance on incorporating accessibility into common evaluation methods, including standards review, heuristic evaluation, design walkthroughs, and informal evaluation with users with disabilities. The Usability Testing section is an overview of usability testing with participants with disabilities.

Preparing for a usability test that includes people with disabilities involves the following considerations:

Ensuring the Facility is Accessible
Preparing Test Materials
Setting Up and Testing the Participants' Configurations
Becoming Familiar with the Assistive Technology
Conducting Pilot Testing

Ensuring the Facility is Accessible

Make sure that the usability testing facility is accessible to participants. Depending on your participants, check things such as wheelchair access into the building, to the test room, and to the bathroom; parking space for a van with a side wheelchair lift; and space in the room for an interpreter to be in the best position. To help ensure that nothing is overlooked, consider the following:

Use a checklist to ensure that you have anticipated any potential barriers. The "Interacting with People with Disabilities" section of the Appendix: Resources lists checklists to help determine if your site is accessible.
Schedule a walkthrough by a person with similar accessibility needs; for example, someone from participant screening who didn't meet the participant criteria.

Provide enough space for a wheelchair, assistive technology, interpreter, personal attendant, guide dog, or other service animal, as needed. Consider using modular tables so that the room can be rearranged.

Preparing Test Materials

Write usability test materials in clear and simple language. This will be especially important for participants with some types of cognitive disabilities who have difficulty processing information or instructions.

Prepare test materials for individual participants based on their abilities and preferences. For example, participants with different types of cognitive disabilities (such as dyslexia, dyscalculia, autism), would do better or worse with different types of rating scales, such as words, numbers, or images of facial expressions.

Be prepared to provide all materials in alternative formats. Materials include directions to the facility, consent form, release form, non-disclosure agreement, instructions for the participant, and tasks to be completed during the usability test. Alternative formats include:

Large print
Braille
Electronic (HTML, plain text, or other format; via e-mail, CD, etc.)
Audio (read live, electronic file, cassette tape)

Note that some people with disabilities don't like PDF format. There are things you can do to increase the accessibility of PDF documents for screen reader users. People with low vision cannot increase the font and print PDF documents, so you will need to provide the PDF document with the font already in the user's preferred font face and font size, or provide the document in a format where the user can set the font themselves.

Plan time to have your documents "brailled" if necessary and remember that you can't make last minute changes to your test materials if you have them brailled elsewhere. Label your braille pages (if they get mixed up you won't be able to sort them out unless you read braille).

Only a small percentage of people who are blind read braille. People who are born blind often learn braille; people who go blind later in life usually don't.

Include alternate format questions in the recruiting screener. Questions about alternative formats to ask participants during recruiting are included in the Recruiting Screener section.

Send consent forms and other documents ahead of time. Sending the consent form, non-disclosure agreement, and any other documents to participants before the test lets them read the documents ahead of time in their own environment, with their own assistive technologies, and at their own pace. Some people may take a long time to read documents and it helps your scheduling if this is done ahead of time. Explain to the participant that some materials, such as the tasks, cannot be sent ahead of time and will be available in the participant's preferred format.

Provide consent forms and other documents for interpreters, attendants, and guardians. In most cases, you will need sign language interpreters, personal attendants, and anyone else who accompanies the participant in the actual test to also sign consent forms, non-disclosure agreements, and other such documents. For some participants with intellectual disabilities, you will also need to get consent from their guardian.

Send materials to interpreters ahead of time. Sign language interpreters can be better prepared if they have at least a rough script of what you will be saying. For formal studies where close translation is important, consider scheduling a little time to work with the interpreter before the test to discuss precise meanings.

Test materials in different formats across participants

For almost all usability tests, it is acceptable to have participants use test materials in different formats, for example, some read tasks themselves and others hear tasks read to them. Just because one participant chooses to have tasks read to them, doesn't mean that the facilitator should read the tasks to all participants. Instead, provide the tasks and other test materials in the format that is best for each participant.

Setting Up and Testing the Participants' Configurations

As previously mentioned in "Consider assistive technology needs" in the Planning Usability Testing section, providing the required configurations can be complicated. When testing software or web-based products, participants might require different versions of different assistive technologies with different configurations. People who don't use assistive technologies might use different system settings, such as large fonts and alternative color schemes.

The Recruiting Screener later in the book includes basic questions about what assistive technology the participant uses. To set up and test the assistive technology, you will need to get specific information about the participant's assistive technology, configurations, and settings.

“Uh… this isn’t the assistive technology I’m used to,” is a phrase that fills my heart with dread. We always check that we have not only the right AT, but also the right version of the AT. For example, there’s a significant difference between JAWS 4.51 and JAWS 6.

Acquire, set up, and test the assistive technologies to the participants' configurations, well in advance of the usability test. Starting early leaves time to address complications, such as acquiring older versions of software, getting different configurations to work on a single computer, or figuring out that different configurations will not work on a single computer and you will need multiple computers for back-to-back participant scheduling.

Becoming Familiar with the Assistive Technology

When participants will use assistive technology (AT) in a usability test, it will be more effective if the facilitator is somewhat familiar with the AT. Otherwise, the facilitator might not be able to understand the interaction between the participant, the AT, and the product being tested (as well as being distracted by the novelty of the AT). Usability test observers, data analyzers, and others will also benefit from having some familiarity with the AT used in the test.

Get experience with the assistive technology as appropriate. You can gain varying levels of experience with AT in the following ways:

Read and watch videos about AT. "Understanding Computer Use by People with Disabilities" in the Appendix: Resources lists many free resources available on the Web.
Get an introduction to and demonstration of the AT from an experienced AT trainer or someone who regularly uses the assistive technology—perhaps someone from participant screening who didn't meet the participant criteria. When possible, have them use the AT with a product similar to what you'll be testing.
Practice using the assistive technology with the product.
After you are somewhat familiar with the AT, ask a regular AT user to answer your questions and help you learn more about how the AT is used with your product.

Conducting Pilot Testing

Pilot testing is especially important when testing with participants with disabilities. There are more things that could go wrong or be new to the test designers and facilitators, such as problems with assistive technologies.

Conduct pilot tests early. If you are new to usability testing with people with disabilities and if you are planning a formal study, conduct at least a couple of pilot tests fairly early in the project so you have plenty of time to work out any issues. If you don't have enough extra participants from your recruiting you might be able to use someone who didn't meet all of the recruiting criteria.

As mentioned previously, you can use pilot tests to help with recruiting because often when participants with disabilities have a good experience, they will tell others. Also, you can use pilot tests to work out timing.

Use pilot tests to work out issues with assistive technologies. Assistive technologies behave differently in different configurations, and users use them in many different ways. Additionally, assistive technologies may require adaptations in your data collection. For example, some screen recording software may conflict with some assistive technology.

Use pilot tests to work out logistics. Pilot tests help ensure that there are no unforeseen barriers or difficulties for participants in the study. For example, in pilot test you may learn that your non-visual directions to your lab can be improved, that you need to find a good place for a service animal to go outside, or that there is a step you hadn't noticed that makes the main path to the lab inaccessible to wheelchairs.

Use pilot tests to work out facilitation. You may need to adjust your facilitation, observation, and data recording. For example, with participants who are deaf you need to mic the interpreter, and with participants who use screen readers you probably want to record audio output.

During a test with participants using screen magnification, some observers susceptible to motion sickness were unable to stay through the entire test because of the motion on the screen. We learned that we needed extra observers to take notes.

The next section, conducting usability testing, provides guidance on what to do during the actual test sessions.