4 ways to combat usability testing avoidance

Phil Barrett — Wed, 20 Jan 2010 20:58:01 +0000

Working with users during the design process will untie project knots and boost team productivity and focus. But there always seems to be an excuse for not testing. Here are 4 ways to counter the excuses and make usability testing happen.

Testing a paper prototype

Excuse 1: “It’ll slow us down”

Finding users, building prototypes and working through hours of research takes time. Why not spend that effort on writing more code?

Counter argument. You say: “Our business objective is to reach profitability as quickly as possible. To do that, we need to understand what our customers really need and make sure we’re all agreed on the direction. A usability test might take some time in the short term, but it will help us reach our overall business goal quicker.“

Usability testing, like many UCD tactics, is an investment. You put in time and money, but you get back a product that sells better and costs less to support. But usability testing is also beneficial during the design process…

The managing director observes a usability test via a video link

1. Design the thing better, quicker: Trying to design a product for target users, without ever meeting any, is like pulling teeth. But if you just watch a few users using a prototype, a competitor product or their current system, they’ll tell you what you really need to know quickly, effectively and (comparatively) effortlessly.

2. Manage the politics more easily: Successful designs come from teams all pulling in the same direction. Usability testing results will reduce squabbles, give confidence to management and get people to focus on improvements rather than feature creep. Even the most sceptical team members can’t ignore videos of 5 or 10 real people battling with their software.

3. Get a team energy boost: Seeing ideas succeed makes the team feel positive. Seeing them fail motivates people to sort things out.

Excuse 2: “Our product is already perfect”

You and your team will become so deeply familiar with the product you’ve designed that you will think it is perfect.

Counter argument. You say: “We believe the product is perfectly easy and useful. But can we prove it? How many problems exist that we’re not aware of? What impact might they have? Developers may think their code has no bugs, but we still hire testers to prove it. What evidence do we have that our design is perfect first time?”

This behaviour is often referred to as “drinking your own Koolaid“. It means you’re doubly ignorant…

You do not know which parts of your design your target users will struggle with.
You also don’t know that you don’t know.

In a thought-provoking piece a few years back called The Five Orders of Ignorance, software engineering expert Philip G Armour says,

“The hard part of building systems is not building them, it’s knowing what to build — it’s in acquiring the necessary knowledge… A functioning system is the by-product of the activity of finding things out.”

Excuse 3: “We already have lots of feedback”

Listening to customer feedback via email, call centre or the web is vital. Analytics and search log analysis is great, too. And it can seem like you’re getting all the user input you need.

A group of developers watching usability testing video

Counter argument. You say: “We’re only getting feedback on major issues and from committed product users – lots of other people encounter our product and never feed back. So we’re getting a skewed perspective. Usability testing will let us observe and discuss all sorts of things that customers and non-customers would never actually feed back about. It will also explain what to do about the strange patterns we’re seeing in our web analytics. This extra insight will give us a competitive edge, because it’s not obvious stuff that our competitors also know.”

Excuse 4: “This concept is not ready to test yet.”

Ready for a usability test

It’s easy to tell yourself that you’re not ready to work with target users yet – that your ideas haven’t settled down to something stable and complete which users will approve of.

Counter argument. You say: “Don’t worry if it’s not ready. We’ll test what we’ve got, and won’t worry much about the areas where we know things aren’t finished. It can give us reassurance that we’re heading in the right direction and stop us from spending loads of time designing a blind alley.”

The truth is, your ideas will never be stable and complete until you’ve had the input from users. Until then, they are just hypotheses. Better to test your hypotheses when they are young and flexible, rather than when you’ve spent weeks on refining them, and publicly declared them as “finished and ready”.

How to run that test

Doing the perfect usability test is no doubt hard. But doing a useful test is really easy…

Pump out a series of pages in Balsamiq or any one of the herd of prototyping tools that are springing into existence.
Set up to record desktop video using Camtasia Studio or Silverback. (Or Morae if you can afford it).
Ask users to tell you stories about using your product or similar products in the real world.
Watch users using competitor products.
Get users to walk through your prototype and listen to what they say (keep pretty quiet yourself).
Summarise findings in a top-down way. What was the overall result? What were the big findings? What do you recommend should be done about them? What were the little findings and what are you going to do about them?
Make video clips of the very finest moments, and encourage everyone to watch at least some of the test videos.

As Bruce Tog says, without iterative usability testing “you’re going to throw buckets of money down the drain”. So just get out there and test.