Even though I only had 12 participants, I learned a lot from my first unmoderated remote usability test.
Use remote testing to find the right things
Loop11’s tools turned out to be really handy for finding the paths people take from point A to B. From the path analysis they gave me, I could see that participants had landed on the success URL 2 or 3 times before marking the task complete, or that they’d passed it up and clicked “complete” on the wrong page. With more results, it would become easier and easier to identify hiccups in the user’s workflow.
Write tasks anyone can complete
To complete my third task, participants would’ve had to know a thing or two about web development, browser support, and HTML specifications. Since I didn’t recruit only web developers, the results I got from that task weren’t very reliable; I couldn’t assess with any certainty whether failures were the fault of the site’s design or the fact that participants had only a vague idea of what I was asking them to do.