You can get the Just Ask book from www.uiAccess.com/accessucd/print.html
Accessibility in User-Centered Design: Conducting Usability Testing
The Evaluating for Accessibility page provides guidance on incorporating accessibility into common evaluation methods, including standards review, heuristic evaluation, design walkthroughs, and informal evaluation with users with disabilities. The Usability Testing section is an overview of usability testing with participants with disabilities.
Conducting a successful usability test that includes people with disabilities involves the following considerations:
- Setting Up the Room
- Orienting the Participant
- Completing Paperwork
- Completing the Tasks
- Collecting Data
- Providing Compensation
Some of the considerations in this section might or might not apply to a given usability test, depending on the participants, the product, and other parameters. The Recruiting Screener near the end of the book lists questions to ask to find out which considerations apply based on participant needs.
See Interacting with People with Disabilities for guidance on avoiding assumptions, asking before helping, talking to and about people with disabilities, and more.
Setting Up the Room
Check that the area is clear. Look for power cords, cables, and wires, which can be especially hazardous to participants with visual impairments.
Right before a participant in a wheelchair came in, I noticed that someone had put a stack of boxes in the hallway leading to the lab and now there wasn't enough space for a wheelchair.
Specific considerations for some participants who are blind or visually impaired
- Don't move anything without asking first when at the participant's site. People who cannot see might run into a chair or other object that is in a different place than they are used to. If you do move anything (with permission), at the end of the usability test ask if the participant wants you to put things back in their original place.
- Record screen reader audio output. Position audio recording equipment close enough to hear the screen reader. When feasible, use a separate audio capture for the participant's voice and for the screen reader. Participants often talk over the screen reader. If each is recorded separately, you can adjust the volumes when analyzing data and making highlight tapes.
- Watch the keyboard. If the participant is using a screen reader, set up the room so that the facilitator can see the keyboard and hear the screen reader. By watching the keyboard you can better tell whether the participant is navigating by using the tab key, or the arrow keys, or just letting the screen reader read through the entire page, for example. Because the focus on the display does not always match what the screen reader is speaking, it might be confusing to watch only on the display. Consider recording keystrokes.
- Adjust video for wide-angle capture. Participants using a screen reader don't need to stay in one position in front of the monitor. Adjust the video camera before you begin so that if the participant moves around a bit, she is not off camera.
- Take speakers and lights for testing at the participant's location. Some screen reader users use headphones and might not have adequate speakers for you to hear the audio. A person who is blind might not have rooms well-lit. Consider taking additional lighting, especially if you are recording video.
One time we were testing at a participant's house, and when the test started there was enough daylight in the room. As the sun set it became too dark to see our test papers and notes. Eventually we asked the participant if we could turn on the room light, at which time we discovered that the light bulb was burned out. Because the participant was blind, he didn't know the light wasn't working.
Specific considerations for some participants who are deaf or hearing impaired
- Provide seating for an interpreter. If an interpreter will be present, provide room for the interpreter to sit in different positions, such as near the facilitator or across from the participant.
- Position seating for a direct line of sight between the participant and the facilitator, for participants who speechread (also known as lip-reading). Encourage the participant and interpreter to adjust their seating to make it easy for them to work together.
- Be sure the room is well-lit so that a participant who speechreads can easily see the facilitator's lips and facial expressions.
- Record both participant and interpreter audio, depending on the situation. If the participant doesn't speak at all, you might want to mic only the interpreter. If the participant speaks some, you probably want to mic both of them. It is usually not necessary to include the interpreter in the video recording.
Specific considerations for some participants who have physical impairments
- Allow enough room for a wheelchair to get in, move around, and be positioned at the computer, table, or other area where the participant will interact with the product.
Orienting the Participant
Encourage the participant to become familiar with the setup of any hardware that will be used in the test, such as keyboard, mouse, and speakers. Encourage them to adjust the equipment, chair, etc. so they are comfortable. If assistive technology will be used in the usability test, allow time for the participant to check the settings and to change them if necessary.
Specific considerations for some participants who are blind or visually impaired
- Introduce yourself as you approach the participant. Don't assume that the participant will recognize your voice. Introduce others who are with you so that the participant knows who is in the room.
- Describe the setting to the participant briefly, including the location of doors and position of the video camera, especially if people will be coming in and out and the video camera will be making noise.
- Explain unusual noises and your activities, such as when you are beginning to record. Most video cameras make a sound when recording starts or stops and the participant might be distracted wondering about the sound. Tell the participant when you or others enter or leave the room.
- Offer your elbow to lead the participant. Don't grab the participant's arm, hand, or cane.
- Give directions about where to be seated. Generally it is best to give verbal directions and not physically direct a person. You can ask the participant if he would like you to put his hand on the chair.
- Tell the participant where there is room for their guide dog or other service animal. At the same time, be aware that the participant and animal might be more comfortable in a different position.
Specific considerations for some participants who are deaf or hearing impaired
- Get the participant's attention before starting a conversation. Say the participant's name in his line of sight or touch the participant gently on the shoulder or arm.
- Take turns talking. A person who speechreads might miss information if two or more people are talking at the same time.
- Be very clear that an interpreter should repeat just what you say. Clarify to the interpreter the importance of translating as accurately as possible, and not adding clarifications.
Specific considerations for some participants with physical impairments
- Don't move mobility aids without asking. Some people who use a mobility aid, such as a walker, might be uncomfortable if it is out of reach.
- Remember seating for a personal attendant as discussed in "Ensuring the facility is accessible" in the Preparing for Usability Testing section.
Completing Paperwork
Participants who were sent consent forms and other "paperwork" before the test, as discussed in "Preparing test materials" in the Preparing for Usability Testing section, might bring it already signed.
Specific considerations for some participants who are blind or visually impaired
- Provide documents in the participant's preferred format, as requested during participant recruiting. Some people read braille very quickly, and some read it very slowly. To help manage time, you may want to ask participants if you can read the documents aloud and provide the braille version for reference. Be prepared for some people to want to use the braille themselves and not have you read it.
We've seen some participants read 2 pages of braille very quickly—much faster than we would have read it verbally, and others take 25 minutes to read the same braille document.
- Be prepared to indicate the place for signature. Some participants who are blind will have a signature stamp. Some participants will have a signature guide, a small device that guides them when signing a document. Be prepared to position a signature guide or straight-edged object so that a participant who is blind or has low vision knows where to sign.
Specific considerations for some participants with physical impairments
- Have a clipboard available to hold documents to be signed. Be ready to hold the consent form and other documents in a position where the participant can easily sign it; for example, a participant with limited upper body movement might want you to hold it up so that she can sign it with a pen in her mouth.
Completing the Tasks
In any usability testing situation, you should provide clear information to participants before the session that:
- Participation is voluntary. They can stop at any time.
- The session is to find ways to make the product better. It is not evaluating them. If they have trouble doing something, that is helping you understand how to make the product work better.
Remind participants of this throughout the session, as warranted. Some participants might forget and need reminders often, such as seniors with memory loss and participants with some types of cognitive disabilities. Some people with disabilities might be more inclined to think a problem is because of their disabilities, instead of the problem being because the product is not designed well.
Communicate thoughtfully about you stopping tasks. Some people with disabilities will be particularly eager to complete tasks without help, and might be bothered by being stopped before they are done if they aren't expecting it. Tell participants at the beginning of the session that you might stop tasks before they are complete. For example, you could say something like, "When we have enough information for a task, we might go to another one, even if you haven't completed the task." Also consider what to say when interrupting a task. Instead of, “In the interest of time I'm going to stop you there,” consider something more like, “Let's stop there and I'll give you something new to do.”
Remind participants that they can stop or take a break. During the session, be particularly aware of the participant's fatigue and stress level. Remind them periodically that they are welcome to stop or take a break. Some participants may be hesitant to ask for a break, even though they need one. In such cases, consider saying that you would like a break.
Be prepared to use alternative techniques for facilitating. Facilitating usability tests and focus groups often involves subtle communication. Some facilitation techniques might not work with participants with disabilities. For example, body language won't work with participants who are blind and with some people with autism; and if a participant is deaf, the technique of just not answering a participant's question during a low-interaction session won't work.
When facilitating a focus group with blind participants, it became apparent that my practiced use of eye contact and body language to regulate the flow of discussion was of no use. I had to devise strategies for communicating the same cues verbally.
Specific considerations for some participants who are blind or visually impaired
- Request screen reader speech rate according to usability test protocol. Screen reader users usually set the reading rate fast and most people who aren't used to listening to a screen reader can't understand it that fast. For an example of fast screen reading rate, listen to the end of the Introduction to the Screen Reader video. When you want to understand specific interaction with the product, you will likely need to ask the participant to slow down the reading rate. Note that while most screen reader users will be happy to work at a slower pace, some might be frustrated working at the slow pace for a long time. For usability tests measuring time-on-task, the screen reader should be set to the participant's normal rate.
Specific considerations for some participants who are deaf or hard of hearing
- Face the participant while speaking, and speak at eye level. Many people who are deaf or hard of hearing rely at least partially on speechreading and thus need to see your lips. Remember not to look down or cover your mouth while speaking. If the participant is seated, sit to speak to him whenever possible.
- Speak clearly. Don't speak too fast.
- Don't just repeat louder. Some people who use hearing aids or cochlear implants are especially sensitive to loudness. If the participant is having trouble hearing your words:
- Try rewording what you are saying. Sometimes a person with a hearing loss might be partially dependent on speechreading because some sounds might not be easily heard even with a hearing aid. Since some words are easier to speechread than others, rephrasing what you said might make it easier for the participant to understand.
- Offer to write down what you are saying.
- Move closer to the participant, while still observing personal space. [1]
Collecting Data
Consider debriefing after each task instead of after the entire test. This is helpful for participants who have spent a significant amount of time on the task, seniors who might have short-term memory loss, and participants with cognitive disabilities who have difficulty processing a large amount of information.
Specific considerations for some participants with speech impairments
- Listen carefully, and ask the participant to repeat for clarification if needed. If you don't understand the participant, ask him to repeat what he said or ask for clarification. Don't pretend that you understand if you don't understand. Wait for the participant to finish, rather than interrupting or finishing his sentence. If it's difficult for the participant to speak, you can repeat the part that you did understand so he only needs to restate part of it.
- Be prepared to offer to use written communication. Some participants might prefer to communicate by typing or writing.
- Be very clear that an attendant should repeat exactly what the participant said. Sometimes an attendant or other person who is used to the participant's speech will clarify what the participant says. That person might be inclined to expound on the participant's comments. Make sure that person knows the importance of saying just what the participant said.
Specific considerations for some participants who are deaf or hard of hearing
- Be very clear that an interpreter should repeat just what the participant signs. Clarify to the interpreter the importance of translating as accurately as possible, and not adding clarifications.
Providing compensation
Specific considerations for some participants who are blind or visually impaired
- State what the currency is as you hand it to the participant if you are paying in cash. People with vision impairments often have a particular way to fold each denomination so they can identify it.
- Verify the spelling of the participant's name if you are paying by check.
We misspelled a participant's name on his check, and he didn't figure it out until he tried to cash it. We had to re-issue the check and he had to make a second trip to the bank.
The next section discusses analyzing and reporting a usability study with participants with disabilities.
References
- Communicating with People Who Have a Hearing Loss. Alexander Graham Bell Association for the Deaf and Hard of Hearing, 1996.