Methodologies

The Session

The idea is not to tax the user too much in the beginning of the session so the initial tasks often surround activities, that while crucial to a product’s success, are familiar to the average user. Generally this is something like a sign-up process.

As I observe the participant completing each task, I probe them with questions like, “What are you thinking now,” or “What made you click there.” The session progresses and a picture of the things that need to be addressed begins to form. After the tasks are complete I confer with the observer and relay any questions to the participant.

Between sessions the observer and I chat and get an idea of what worked and what didn’t, where issues were popping up and if any adjustments need to be made.

After the day is complete I aggregate all of the data and present it, along with video segments of the sessions, to the stakeholders in the project and schedule a follow-up meeting. The follow-up meeting is a great chance to go over the sessions and the conclusions reached in my research and to lay out a road map for further testing.

All of this happens in a relatively short period of time which lends itself to augmenting a rapid, iterative development cycle.

Live Usability Testing: Mobile Devices

I employ a custom-built mobile usability sled to record mobile app testing and Silverback to record and annotate. An observer is usually in another room or off-site, watching via Skype or another screen sharing platform. The sled allows the respondent to hold the mobile device as they normally would so as to make the process as close to actual use as possible. At the conclusion of the session the observer can then relay any questions they’d like to ask the respondent.

Live Usability Testing: Desktop/traditional web

Depending on the client’s needs, I use either Silverback or Morae to record and share the screen for the observers. Typically this is done with a laptop and monitor with the observers being able to watch and comment at the end of the session.

These two techniques all the moderator have a more personable experience with the respondent and allow for a better assessment of subtle clues that can’t be ascertained without being present in the room.

Remote Live Testing

Platforms like GoToMeeting and others for a set of tools that allow a researcher to conduct a session with the respondent in the comfort of their own home. This can provide a greater geographical reach for the pool of respondents which can help to combat the “localization” effect, ex. cultural differences toward a product in New York versus Houston. While the researcher is not present in the room and thus may miss out on some subtle non-verbal reactions, the ability to see the respondent and vice versa is very effective.

The major downside to this method is that even with the most tech savvy respondent there are often technical issues that will arise when attempting to start the session such as browser incompatibility, bandwidth issues and others. In order to account for this higher rate of difficulty, sessions are generally shorter and more are scheduled for the day.

Remote Testing

Projects sometimes call for the use of a remote platform like UserTesting.com. This allows the participant to go through a series of pages on a client’s website and attempt to complete tasks in order to test certain assumption about its design. This method is what I consider the best way to introduce a team to the concept of UX research as the barrier to entry is very low. That being said, the findings will not be as rich or actionable as live testing but help provide direction and insight to base further full testing upon.