This week, the assignment in my Usability I class at Kent State was to moderate a usability test.
Preparing for the test
To get ready for the test, I read over the script and tasks provided by the instructor. The introduction script was straight from Steve Krug’s Don’t Make Me Think, and we were encouraged to make it our own. I decided that for this first test, I would just read it and see how it felt. (It was kind of awkward; I’ll definitely be paraphrasing next time rather than reading it straight from the page.)
To set up my home iMac for testing, I created a new user account with Parental Controls enabled, so that the only things the user could open would be the browsers, Silverback (used for recording, more on that in a second), and basic OS functions like Launchpad and Mission Control.
I used Silverback 2 to record the test, and it was just what I needed. Very straightforward, with added goodies like highlighting and task segmentation via the Apple Remote. Although I doubt that I’ll be doing many more usability tests from my home computer, I am eager to see what Silverback 3 has in store.
My first participant
My long-time friend and down-the-street neighbor Andrew Medhurst was kind enough to participate. Andrew is a software engineer with a Game Design degree. He has done some testing through UserTesting.com before, so he was very good at remembering to think aloud the whole time, and in general it was enlightening watching him perform the required tasks.
In a real-life situation, though, I would probably have screened Andrew out for this particular study. I’m not sure whether it was my lack of skill in asking the right questions or design and programming experience, but his thinking-aloud monologue tended to be more like a critique of the design than a play-by-play of his motivations and observations. (I suspect that most designers would be that way, myself included.)
Watching the replay
Playing back the video, I noticed several areas where I need to improve my moderating skills.
First, as I mentioned earlier, I was reading straight from the script most of the time, which in some places sounded awkward and dispassionate, and in others, almost mocking or sarcastic. Since the script wasn’t written by me, the words weren’t the exact ones I would’ve used, and it was fairly apparent. Next time I will definitely take more time to paraphrase and memorize the script.
Several times during the test, I found myself drawing a blank when it came to asking the right questions or giving Andrew the right feedback to keep him on-course. It really is a balancing act, walking the fine line between saying things that won’t subtly bias the participant and making sure the think-aloud data is relevant.
Finally, I definitely need to work on my poker face. I found myself smirking, wincing, or making this face uncontrollably when Andrew was hard at work trying to figure out how to complete a task. Luckily, he didn’t happen to see me at those times, but if he had I doubt he would’ve acted and thought the same way afterward. (Well, maybe he, who’s known me for so long, would’ve been okay. I imagine anyone else would’ve been offended or self-conscious, though.)
All in all, it really did go pretty well. More than anything, I am glad that I got to practice in such a safe, controlled setting before conducting a test for a real client or employer—I am sure moderating a study is stressful enough when it’s not your very first one.