Rita Sembajwe, Health Informatics Specialist, BreathEasy Project Manager, RTI International
Sarah Cook, Survey Methodologist, BreathEasy Usability Testing Lead, RTI International
We just completed usability testing for both our Android smartphone application for patients and our clinician dashboard. Overall, it was a success and we received great feedback on the look, feel and functionality of both technologies. Equally as important, we learned a lot about what we would need to consider the next time around as we recruit for and carry out usability testing sessions.
When we were recruiting for usability testing sessions, we needed to be more prepared for patient no-shows. Usability testing is dependent on the turnout and participation of the relevant testing audience. You might think to plan for one or two no-shows, but what could you do to prepare for a 50 percent turnout rate? How could you encourage the participants to show up?
Although our patient usability testing session slots filled up quickly when we were recruiting, we had trouble getting at least half of our patients to turn up for the actual sessions. At the last minute, we managed to reschedule and interview a few, but the effort was an unexpected undertaking. In retrospect, we now recognize several factors we needed to consider:
- Our patient population may have higher odds for no-shows due to possible depression and anxiety co-morbidities that influence their behavior.
- There may be challenges with public transportation and finding the session locations.
- Participants may not understand exactly what usability testing entails.
We now realize that there are a few things we could do proactively to increase attendance rates in the future. We could:
- Follow up with participants more frequently after they agree to participate in the sessions.
- Hold sessions in the most convenient locations.
- Increase the incentives for participation.
- Identify back-up participants in case we need additional people at the last minute.
The usability testing session for the smartphone application also taught us that we must schedule time for a crash course on how to use the smartphone device prior to or at the beginning of the usability testing sessions. Such planning would ensure that the usability testing sessions focus more on testing the application and less on learning how to use a smartphone.
Finally, we believe it would benefit us to incorporate a review of the patient smartphone application into the clinician dashboard testing session, preferably toward the end. From our interactions with clinicians, it is clear that they would appreciate seeing the technology their patients would be using to submit ODL data, and that it would offer them a greater understanding of how the patient-generated data translates into feedback for them on the clinician dashboard.
We learned many lessons — all worth consideration. We plan to prepare for all of these the next time around!
