Seven Common Usability Testing Mistakes

Since we are in the process I thought this would be a good article to review. Seven Common Usability Testing Mistakes discusses how many usability problems stem from errors in design and organization. The author, Jared M. Spool, travels to work with many companies and has observed theses mistakes regularly. I guess you could say he tested the usability testing process.

1. Do You Know Why you're Testing? This is difficult question to answer. I know we discussed this answer quite a bit when designing our test plan. He discusses that testers often use the tests to determine how users 'feel'. That is not usability, that is user experience. The main focus of the testing should be when and where the user begins to feel frustrated because they are unable to complete the task. I completely agree. What the participant feels is important in these tests but I see it as a secondary measure. Record the feelings but view them as a side-effect of the procedure.

2. Not Bringing the Team Together. This to me this goes without saying. If everyone isn't there paying attention, then not everyone will be on the same page. When you are not on the same page, communication suffers. I like his comparison the the 'Telephone'. Played that a lot when I was a kid.

3. Not Recruiting the right Participants. We discussed this in class and I believe we all agreed that friends and family members where not good choices. The ideal choices are those who fit the demo and come in to the process cold. I like what he says here about what can happen with the wrong participants, do they know too much so they won't experience the problems real users would find, is one issue. The next contradicts what we discussed to a degree. Spool says focusing too much on demographics and ignoring individual subtleties could lead to false conclusions. He says you should focus on user attributes like behavior. How will the users each behave and what attributes cause this distinction? I like that, too. However, I think it will narrow your field too much and become too niche oriented.

4. Not Designing the Right Tasks. This is huge. I had several problems trying to design tasks the first time I tried it in my Production Team class. And I fell into the issues of 'leading the participant', using the same language, etc. Spool touches on the first thing I thought of when considering testing, why wouldn't the user go to the search box? In a test he helped with for Ikea, this is exactly what happened. They were asked to find a bookcase, so they ALL went to the search field and typed in 'bookcase'. So the task was changed to become more of a story telling them what they were going to put in the bookcase, how much stuff they had and they called is shelving, not bookcases. This is the approach I would like to use. When originally wrote the tasks in the class i mentioned before, I tried to do that. 'You have an 8-year old daughter and you to register her for swimming lessons. Find the form.' I was half-way there. It is completely accurate that task design will effect your results. I don't see how this could be ignored. 'Context of use'. I will remember that and use it.

5. Not Facilitating the Test Effectively. Spool says the facilitating is learned skill. He says he has never met anyone that was adept to it. A good one will draw the right information without giving away the goods. They keep it interesting not only for the participant, but the observer as well.

6. Not Planning How You'll Disseminate the Results. Here he focuses on the getting the results to the right people so that action can be taken.  I would agree that reports are often not read. You may give them out, but they more than likely will not get read. He proposes review sessions after each test, an email discussion list about the test and interactive workshops. I like these ideas! I think theses are great ways to make sure everyone gets the results.

7. Not iterating to Test Potential Solutions. Although testing may illuminate problems, it fails to tell us how to solve them. There are several solutions for any given issue, but which one do you use? You will need to test again. Spool says paper mock-ups will work for this round. Do not cut corners. If your 'solution' is more confusing than the original problem, it could end in disaster. Follow-up testing is a must to make sure you have chosen the right conclusions.

I learned a lot from this article. most of it we discussed in class but it nice to read how these things work in the world.

What's In A Number?

A lot according to who you talk to. In fact, the question reminds of the fablesd, "how many licks does it take to get to the center of a Tootsie Roll Tootsie Pop?". The wise answered, "Three".

Looking of the world of sample testing, there are several views how many participants are enough to get a valid result. Let's start with What’s in a Number? By Carol M. Barnum. She states, "Whereas 7 (plus or minus 2) is the mantra for structured writing and other methods for organizing information, 5 (plus or minus 2) is the mantra for the number of participants needed in a usability test." They added that 5 will yield 80% of findings, although now they have lost faith in this number when it comes to testing Web sites. They said it will take more than 5 to uncover all the usability issues.

Jakob Nielsen thinks it should be more segregated into four different areas: heuristic, usability, participants and scenarios to gather a more comprehensive view of the web site. Nielsen says that a substantial portion of the user population should be used to correlate your findings.

The article also mentions combining methods when testing. I like this idea the best. I am a firm believer in mixing-and-matching to get the most of everything. This is what I will do.

iPad Usability: First Findings From User Testing

iPad Usability: First Findings From User Testing
by Jackob Nielsen

"It looks like a giant iPhone," is the first thing users say when asked to test an iPad. (Their second comment? "Wow, it's heavy.")

This is the opening statement of the article and it is exactly what I thought, too. The first time I held one, I also thought the second comment. I still have yet to find a way for the iPad to fit into my lifestyle.

Overall, I think certain assesments should have been obvious. The primary issue being that the main interface has not been customized to fit the larger screen. Given that the main screen sets the tone for the the other apps, I would have thought this would have been the most important interface to get right. Nielsen says that with the larger screen, the focus is further away from the tabs at the bottom of the screen and more difficult to discern.

He also discusses 'wacky interfaces'. Since there are no standards an no expectations, anything you can show, can become an interface. As a result, there are inconsistent interfaces. The examples he gives are:

In different apps, touching a picture could produce any of the following 5 results:

• Nothing happens
• Enlarging the picture
• Hyperlinking to a more detailed page about that item
• Flipping the image to reveal additional pictures in the same place (metaphorically, these new pictures are "on the back side" of the original picture)
• Popping up a set of navigation choices
  

Without rules, anything goes which will create confusion from app to app. This amounts to confusion resulting not finding the interface, too many varieties are harder to remember than a standard set and the user can activate an interface unintentionally and create even more confusion ('How do I get out of this?' or 'Where am I?').

I absolutely agree with his summary:

iPad apps are inconsistent and have low feature discoverability, with frequent user errors due to accidental gestures. An overly strong print metaphor and weird interaction styles cause further usability problems.