These figures tell us a number of things;

• Retailers with strong brands can still gain sales by entering the online market – customers expect them to be there, so even late entrants such as House of Fraser can make progress.
• The greater your brand reach, the greater your chance of making sales in a tough market. People expect to have choice and convenience. Online-only brands will struggle unless they have a true point of difference in a fiercely competitive market.
• Retailers who really understand their customers will succeed in a fierce market.

I have spent many years working in marketing departments of retailers and in stores, and I have never spoken to a retailer who would ever say they don’t know their customers. They must do – customers walk through the doors in their hundreds of thousands each week. They speak to staff at tills, on shop floors, by phone, via e-mail, on doorsteps and in focus groups, every day. Market Insight teams carefully examine basket data from tills, loyalty cards and web analytics. There has never been more data on what people are doing in stores, online or over the phone.

For many years retailers have prided themselves on their ability to second guess what a customer will respond to. How they should lay out a store, what to merchandise by the till, the front door, on the home page or at a category level on a website. They think about which tools will be useful, which image is right and which promotion is best.

Ever better, retailers carry out multi-variate testing to find out what works best, they test press ads, TV ads, e-mail campaigns and direct mail shots. They can prove which version works best, and back the winner.

But do they know why?

In the course of my retailing career, I put together successful promotions, advertisements and product launches. I was even involved in some that were not so good. For all I would be able to tell you why I thought they worked or had failed but I could never actually prove my theory. Did we hit upon a lucky idea, or find the secret formula? If so, could we re-create it for a new product, different category or new season?

The answer to this question lies in talking to customers, observing their behaviour and listening carefully to what they tell us. When done properly, this can give real insight into the most important question: why?

Can I repeat that, yes, like many retail professionals my experience and skill meant I could get it right more times than I got it wrong, but is that enough when we face tougher trading in 2010 than most of us have ever seen at any time in our careers?

Do you know how much it costs to talk to your customers and what the returns could be? Here at Flow, we do and I know you would be surprised.

The dissertation looked at how control room staff at London Underground’s Victoria Line made sense of the information they received about on-the-ground events through cues from their equipment and from each other. Using field observations, the study uncovered the situations that gave rise to sensemaking, the strategies adopted to ease and speed up sensemaking and the bottlenecks in information-seeking.

The Ulf Aberg award, given annually, recognises outstanding Masters projects in Ergonomics. Projects are assessed on the quality of the research, expertise, originality, clarity and interpretation of results.

Ulf Aberg, after whom the award is named, began his career in 1961 and spent over a decade working with Ericsson and later with the National Defence Research Institute. Aberg co-authored the first Swedish textbook on ergonomics and was the first foreign member of the Ergonomics Society and the founding chairman of the Nordic Ergonomics Society.

Congratulations to Frankie!

Testing a paper prototype

Excuse 1: “It’ll slow us down”

Finding users, building prototypes and working through hours of research takes time. Why not spend that effort on writing more code?

Counter argument. You say: “Our business objective is to reach profitability as quickly as possible. To do that, we need to understand what our customers really need and make sure we’re all agreed on the direction. A usability test might take some time in the short term, but it will help us reach our overall business goal quicker.“

Usability testing, like many UCD tactics, is an investment. You put in time and money, but you get back a product that sells better and costs less to support. But usability testing is also beneficial during the design process…

The managing director observes a usability test via a video link

1. Design the thing better, quicker: Trying to design a product for target users, without ever meeting any, is like pulling teeth. But if you just watch a few users using a prototype, a competitor product or their current system, they’ll tell you what you really need to know quickly, effectively and (comparatively) effortlessly.

2. Manage the politics more easily: Successful designs come from teams all pulling in the same direction. Usability testing results will reduce squabbles, give confidence to management and get people to focus on improvements rather than feature creep. Even the most sceptical team members can’t ignore videos of 5 or 10 real people battling with their software.

3. Get a team energy boost: Seeing ideas succeed makes the team feel positive. Seeing them fail motivates people to sort things out.

Excuse 2: “Our product is already perfect”

You and your team will become so deeply familiar with the product you’ve designed that you will think it is perfect.

Counter argument. You say: “We believe the product is perfectly easy and useful. But can we prove it? How many problems exist that we’re not aware of? What impact might they have? Developers may think their code has no bugs, but we still hire testers to prove it. What evidence do we have that our design is perfect first time?”

This behaviour is often referred to as “drinking your own Koolaid“. It means you’re doubly ignorant…

In a thought-provoking piece a few years back called The Five Orders of Ignorance, software engineering expert Philip G Armour says,

“The hard part of building systems is not building them, it’s knowing what to build — it’s in acquiring the necessary knowledge… A functioning system is the by-product of the activity of finding things out.”

Excuse 3: “We already have lots of feedback”

Listening to customer feedback via email, call centre or the web is vital. Analytics and search log analysis is great, too. And it can seem like you’re getting all the user input you need.

A group of developers watching usability testing video

Counter argument. You say: “We’re only getting feedback on major issues and from committed product users – lots of other people encounter our product and never feed back. So we’re getting a skewed perspective. Usability testing will let us observe and discuss all sorts of things that customers and non-customers would never actually feed back about. It will also explain what to do about the strange patterns we’re seeing in our web analytics. This extra insight will give us a competitive edge, because it’s not obvious stuff that our competitors also know.”

Excuse 4: “This concept is not ready to test yet.”

Ready for a usability test

It’s easy to tell yourself that you’re not ready to work with target users yet – that your ideas haven’t settled down to something stable and complete which users will approve of.

Counter argument. You say: “Don’t worry if it’s not ready. We’ll test what we’ve got, and won’t worry much about the areas where we know things aren’t finished. It can give us reassurance that we’re heading in the right direction and stop us from spending loads of time designing a blind alley.”

The truth is, your ideas will never be stable and complete until you’ve had the input from users. Until then, they are just hypotheses. Better to test your hypotheses when they are young and flexible, rather than when you’ve spent weeks on refining them, and publicly declared them as “finished and ready”.

How to run that test

Doing the perfect usability test is no doubt hard. But doing a useful test is really easy…

As Bruce Tog says, without iterative usability testing “you’re going to throw buckets of money down the drain”. So just get out there and test.

The brief

The British Association and College of Occupational Therapists (BAOT/COT) is the national professional body for occupational therapy students and staff in the United Kingdom.

BAOT/COT is responsible for setting professional and educational standards, advising on policy, and supporting its 29,000 members’ research and development, professional practice and Continuing Professional Development (CPD).

BAOT/COT’s main channel of communication to members is their website. As a key component of their service to OT staff and students they wanted to ensure they were providing a valuable and easy-to-use resource, so they asked Flow to help them understand what members and non-members wanted from this resource, and then to re-design the site around these needs.

What we did

Using a range of research techniques throughout the project, Flow investigated the needs of Occupational Therapy staff and students, and designed a new website for BAOT/COT around those needs.

We interviewed Occupational Therapists, OT support workers and students to understand their perceptions of BAOT/COT as an organisation, the BAOT/COT website and other resources they use to aid them in their studies or practice. We synthesised the insights gathered from this research into a series of personas, each illustrating characteristics of different members of BAOT/COT’s audience; and used these personas throughout the project to guide and evaluate design decisions.

BAOT/COT had a wealth of information on their existing website, however, as the site had grown organically this information had become increasingly challenging to locate. To ensure content was well organised and easy to locate we undertook a card sorting exercise with people representative of BAOT/COT’s audience. This enabled us to identify the different mental models people used to understand the content that BAOT/COT wanted to include on the new site, and guided the site structure and labelling of content.

Moving into the design phase of the project, we used the insights from our initial research activity to guide concept generation and development. In order to validate our design decisions, we also tested mock-ups of the site with users at every stage – from initial concept sketches through to the final visual design, ensuring we were developing something which met users’ needs and satisfied BAOT/COT’s objectives.

Once the new site was built, we tested it again with users to validate the final designs and evaluate the site against the original brief.

The results

BAOT/COT successfully launched their new website which has been designed to grow organically, forming the foundation of a continually improving member resource.

BAOT/COT’s new website has enabled them to create a more engaging and valuable resource for members and non-members alike. Having moved away from a website which pushed information to members, to one which stimulates online debate between the organisation and its members, BAOT/COT anticipate an increase in membership and in the number of people using the site.

www.cot.co.uk

“Flow's user centred approach helped BAOT/COT understand our users' online behaviours and needs. It also helped clarify our own business goals. Clear user priorities emerged from Flow's research with our members which helped us make confident decisions about site structure, design and navigation. The team at Flow were quick to understand our values and aspirations as well as the practical challenges we faced. Their approach inspired confidence and trust. The new BAOT/COT website which emerged from the project continues to add more and more value to our business.”

Stephen Little, Web Manager and Editor for the British Association of Occupational Therapists and College of Occupational Therapists

There are a number of reasons I don’t love my phone (and a few reasons it’s not so bad):

1) I can’t walk down the street writing a text message

Ok, I like writing text messages and I like to send a quick message every now and again. I am also female and I like to multitask. With my old phone (Nokia N95) I used to be able to walk down the street, not looking at my phone and feel the buttons and know what I was pressing and write a message. With my nice smooth-screened iphone I have no clue where the buttons are and lampposts keep jumping out at me so the quality of the experience and my efficiency has decreased.

2) It feels like I’m putting unnecessary stress and strain on my thumbs while I try to hover to write text

Maybe it’s just me (as a 2 thumbed writer of text) but when I type any text into the iphone I find that my thumbs are kind of hovering over the keypad and I take more strain on them. If I’ve been taking lots of notes or writing longer messages I feel my thumbs starting to get tired. I can’t seem to find a nice resting point on the phone without activating a key.

3) I feel like I have to be a robot and hold the phone just at the right angle or it keeps switching between the different views

Now I don’t tend to hold my phone particularly straight when I’m using it, but as a traditional girl I prefer my phone to be upright (in portrait view) rather than sideways (landscape). However, on my iPhone, if I’m just casually looking it, reading something perhaps, then the screen has a tendency to just switch without asking to landscape view. All I do is casually hold it at a comfortable angle in my hand. Now, if I was a robot, then everything would be at neat 90 and 180 degree angles, and I wouldn’t have this problem. But I’m not a robot. (Someone told me how to get it back to portrait, but it doesn’t seem to work all the time).

4) I have to keep the phone switched on for my alarm clock to go off in the morning

I’m the type of person who likes to go to sleep with their phone switched off and charging overnight. Now, I know that the iPhone isn’t capable of this, although I have no idea why. However, back to my alarm - if I switch my phone off it doesn’t wake up automatically and switch itself on and then wake me up. Very rude if you ask me. So now, if I’m to use my alarm I am forced to keep the phone switched on. I know silent exists, so I shouldn’t be disturbed in the middle of the night by incoming calls or texts, but if the phone is there, and I wake up in the middle of the night I might sneak a quick peek to see if I’ve received a text overnight, so, I prefer to have it switched off. And I don’t want to lose precious battery life if my phone doesn’t need to be charged. I also liked the way my old phone would tell me how many hours I had to sleep, something my iPhone doesn’t do either.

5) It takes longer to send text messages

One of my ex Flow colleagues Martin once did a study looking at how fast people could send text messages on different devices. There was a difference between the speed that they perceived they were typing the message and the speed that they actually did type the text message. I haven’t measured myself, so maybe it’s true for me too (although I think not if you know me and have seen me text).

T9 was great. Nice and easy if you’d learnt it, with an option to scroll through the words that might come up as possible combinations. Worked fine. Now I have a qwerty keypad and no choice to revert to the old numeric keypad T9 that I love. I was hunting around for the option to switch this on on my iPhone but there doesn’t seem to be one – of course I didn’t get a manual (as it relies on me being motivated to go and seek one online) so I haven’t been able to check that. Now, I can touch type on a large keypad so I’m pretty aware of where the buttons are on a keypad but having to use them with my thumbs causes no end of problems. I’ve got quite large fingers and thumbs for a girl, but still I’m constantly pressing the wrong key, switching to capitals when I didn’t want to, trying to work out how to not accept the word suggestion it offers baffles me. Though I do like the way that the recommended words account for the fact you probably pressed the wrong keys.

6) I don’t know how many text messages I’m sending

Now, for those of you who aren’t aware, not all phone tariffs have unlimited texts. Therefore there is a big difference in price between a text that is 160 characters long and one that is 161 characters long. My iPhone seems to disregard this, and doesn’t tell me how many characters I’m using. Is the assumption that we all have bottomless pockets or that the number of text messages flying across the world should increase? A simple indication of the number of texts I’m creating would be a real bonus.

7) The battery life is bad – I refuse to keep it connected to an energy source all day

Ok, if you don’t use the phone then it’s fine. But as it is, there to be lots of other things on the iPhone aside from the phone – and it’s designed for these to be used better than the simple phone functions (as far as I can tell). It seems a real shame that there isn’t a low energy mode that will conserve power, or I can’t switch the thing off while I’m powering up overnight.

Things I love about it

1) It’s nice to have all texts viewed as a conversation

This is a really nice touch. I’d like to have the ability to order the text conversations alphabetically by person though, so I can easily find a previous one with a certain person, but then I’d have to be able to short link to a letter in the alphabet but as there isn’t a keypad, I can’t seem to do this – hmm…

2) It’s  nice to be able to view my voicemails and play them back

A while back at Flow we were designing a new voicemail system, and people we spoke to said they found great value in voicemails from loved ones, children who lived far from home and relatives who’d passed away, so the idea that the messages live on your phone and you can replay when you like – even on the tube – is a great one.

I’m sure there are more things that I love, but if I was to include 7 here that would seem unfair to my rant.

Back to the Roots: If email is the past, is Google Wave the future?

Ex-Flowster Johanna Kollmann, now doing great things at Vodafone, shared her experience of using Google Wave with a tightly-packed audience. Her main argument was that Wave is a great advance on email, offering us something much closer to natural, oral communication, but with the advantage that it can be stored and traced.

Now, there’s a discussion to be had about what constitutes “natural” communication, and whether what we consider to be natural is just the result of using technologies that we are more used to. But we didn’t manage to have that discussion on the day.

Instead, Johanna gave a demo of Wave, and then took some questions. Though much of the discussion focused on the details of the interaction design (which still seems to have a few kinks to iron out), several people said that they didn't “get” Wave. The problem seems to be that by combining the most useful features of email, instant messaging and virtual conferencing tools, Google may have created a product that, for all its advantages, confuses some people by not being immediately recognisable as one thing or another.

Ground-breaking new products can be baffling at first to people whose expectations are formed by older paradigms, but when we use them they begin to make sense, and we gradually accept them and change our behaviour accordingly (think Twitter, or for those with longer memories, the mobile phone). But on the other hand, some new products are insufficiently well-defined at the proposition level (that is, nobody can quite define what they are for), and our research shows that this inevitably has a direct negative impact on the experience of using them.

It remains to be seen which of these two possibilities applies to Google’s Wave, but I’m impatient to see which one it is.

Johanna’s Slides are on her blog, here.

X-Ray Listening

Judy Rees, co-author of Clean Language showed us how she teaches people to listen better, using techniques developed in Cognitive Linguistics and Psychotherapy.

I won’t attempt to explain Judy’s method in detail here, as I’m not sure I can do it justice (and, ahem, because my notes aren’t that detailed), but in a nutshell it is a way of combining template questions with the respondent’s own words, to produce endlessly adaptable, open questions. So for example, a template question might be “Is there anything else about...”, onto which the questioner adds a key word or phrase used by the respondent themselves.

After a brief introduction Judy took the group through an exercise. We broke up into pairs, identified some relevant topics for investigation, and took turns at asking questions and listening to each other’s answers. First we did this spontaneously, using our own choice of words, but the second time we used Judy’s Clean Language technique to frame our questions.

The results were striking: everyone said that they felt more comfortable and more ‘listened-to’ when answering the ‘clean’ questions, compared to the spontaneous ones. On the other side, the questioners said that the ‘clean language’ made it easier to formulate the questions on the fly, and elicited more detailed, more honest answers.

At Flow, we spend a lot of time talking to people, trying to ask the right questions, and trying to listen. Most of us find that scripts are too rigid, so we use semi-structured discussion guides to keep us on the right topic, but we formulate or questions spontaneously, using a variety of ad-hoc rules and best practices to get the best results. We are always looking for the best ways to make people feel comfortable, while still getting the freshest and most honest nuggets of information from them.

This brief introduction to Clean Language showed that it is a potentially useful technique for improving both the quality of interview data and the efficiency of the interview process, all the while making respondents feel more at ease. A win-win-win scenario, if I’m not mistaken. I look forward to finding out more about this, and trying the techniques out in a real interview. I’ll let you know how it goes.

The next article in the series is here.

It’s similar to the idea in ethnography that the very presence of an observer will lead to modifications and unnatural behaviour.

There is also the idea that people may not have conscious access to the real reason for their behaviour. In trying to explain their actions to the moderator they will introspect and provide an answer that they feel is rational, but is effectively made up.

Malcolm Gladwell sums this up something like this (I’m paraphrasing here): Basically—we feel about a thing, then act.

And then, the moderator asks us why.

Faced with this question, we try to think up a plausible, rational-sounding explanation for our actions. And then—here’s the thing—we alter our future behaviour to match that rationalised thinking.

Wilson and Schooler investigated this phenomenon in depth, concluding: “We come up with a plausible-sounding reason for why we might like or dislike something, and then we adjust our true preference to be in line with that plausible-sounding reason.”

So if we had never been asked why we did something, we might continue doing things differently.

However, without entering into conversation with a participant we can only say what happened; and with no insight as to ‘why’ we can’t make decisions and can’t improve.

The problem then is that we need to know ‘why’ but can’t ask ‘why’.

Here’s a few methods that we use at Flow:

Sometimes more open interviewing will tell you what you need. If you need to know why a participant clicked on that link (or didn’t), questions such as “Tell me about the kinds of things you have looked for in the past on a site like this?” can tell you about the keywords or visual elements that a participant is searching for.

“What are you interested in finding out at the moment on a site like this?” can tell you what it is they haven’t found yet.

Ann Light continues in this vein: “An undesirable, but common, way of interrupting evocation [the flow of recall, in this instance] is to invite the interviewee into a judgemental mode. To avoid this, there is no use of questions starting ‘Why... ?’ Instead, carefully manipulated ‘How... ?’ and ‘What... ?’ questions cover the same ground: ‘How did you know that X?’ ‘What were you thinking at the moment when X?’ This does not interrupt the recounting process. So ‘tell me how it was that you came to be looking for this site that day’ does the work of ‘why were you looking... ?’”

Of course together with the different ways of asking why, the facilitator needs also to combine high degrees of empathy and observation. Interpreting what the participant does and says, and ultimately understanding those things will enable us to make better decisions about what to do next.

References

Wilson and Schooler (1991) Thinking too much: introspection can reduce the quality of preferences and decisions, Journal of Personality and Social Psychology 60 (2), pp181-192
Light, A (2006) Adding Method to Meaning: a technique for exploring
peoples’ experience with digital products, Behaviour & Information Technology 25 (2), pp175-187

If you want to find out how well your website navigation structure works for your customers, Treejack is a great tool for the job.

On the other hand, if you want to know why certain parts performed poorly, and what to do about it, you’ll need to get inside the head of your customer. The tools for this are your eyes and ears.

Treejack was developed in New Zealand by Optimal Workshop so has been built with a user-centered approach in mind.

It’s a tool to test the navigation structure of your website. Treejack will pinpoint the most difficult areas or items to find, based on click-trails as survey participants navigate through a prototype of your website’s structure. (the prototype is a simple “tree” of text links generated from a spreadsheet you paste in‚ it couldn’t be easier)

Treejack is a great tool, saving time and headspace, but it is no silver bullet.

You’ll get summarised and detailed outputs showing where each participant went, how directly and quickly they found set items during the survey. But it won’t tell you how much sense it made to them, or why the tricky areas were confusing.

To design a website that’s intuitive to navigate it’s essential to understand how your customers will interact with it. There is simply no substitute for observation when it comes to gathering these insights.

Teaming Treejack up with qualitative one-on-one research makes a killer double-act bringing you the best of both worlds.

Some tips for integrating Treejack into user research sessions:

• Run a warm up exercise on a generic “tree”. Clicking through a bare-bones navigation is quite abstract so this helps participants get used to the interaction style.
• Encourage participants to “think aloud” while using the prototype. When you notice them pause, they’ll be thinking. Having them vocalise their experience is the closest you’ll get to knowing what’s behind their thoughts and any indecision.
• Save your questions till after each task. Interrupting the participant mid-flow can make them change their behaviour, skewing the Treejack report. Let them click through naturally then discuss it afterwards. You’ll need to rely on your note taking here.
• Have a duplicate Treejack survey open in another tab. This way you can ask participants to re-trace their steps without affecting the Treejack results.
• Ask the participants to “rate” each task for how much sense it made to them. Treejack shows where and how they found an item, but doesn’t tell you whether this made sense to them.
• More participants, fewer tasks. As people develop a familiarity with the “tree” they will start memorising where things are, making your findings less useful.
• Use your eyes. The old adage, “it’s what they do, not what they say” is as relevant as ever here.

I’d be interested to hear about anyone else’s experiences. Go check it out at www.optimalworkshop.com/treejack.htm

(an earlier version of this article was published on http://www.userexperience.co.nz/)

Flow frequently uses ethnographic research methods to gain a deep understanding of the social and working lives of people who use different products and services in different contexts. The findings provide richer insights into service and product design requirements and opportunities for innovation, particularly when designing for global and multi-cultural audiences.

One of the key questions around ethnographic research is how its findings are transformed into design. One example of such a process was presented at a recent UX Brighton: ‘A Journey from Ethnography to Design’. The event included two speakers: Simon Johnson, User Experience Consultant at Flow and Miles Rochford from Nokia. Simon spoke about the ethnographic research and subsequent design that he completed for the Environment Agency. Miles’ presentation focused on using ethnography to design products for emerging markets.

The Environment Agency commissioned Flow to conduct contextual research and subsequently design an interactive map that will provide users with coastal erosion information – a national project that will affect 2.1 million houses on the coast. The key objectives were to establish what an erosion map should look like, how it should work and what sort of information should accompany the map.

Claire Mitchell, Flow’s Principal Consultant on the project and Simon started the project with a research phase that included ethnographic field observations in two coastal settings: Norfolk and Hastings. Simon spent two weeks documenting the lives of coastal communities, interviewing local people and immersing himself in their lives. Additionally, Simon interviewed eight professionals at Flow’s experience labs in London.

Ethnography enabled Simon to apply his empathy and humanistic values to drive the project. It was clear that his findings provided the Environment Agency with a rich understanding of the concerns, information needs and myths that people who live in rural coastal communities might have. Simon described how his research findings confirmed some of EA's current thinking, provided new insights and defined the subsequent design process and deliverables.

The research that Claire and Simon conducted described how emotive the coastline is, an institution in British history that invokes strong feelings and forms a strong part of a shared heritage. The implications were the need for the Environment Agency to communicate that it cares and to reassure people that action was being taken to protect the coast. It was also clear that people trust locals and distrust central government, erosion maps caused alarm and that a certain amount of local knowledge derived from  ignorance and/or myth. An example of a myth was the commonly repeated argument that the government was making money dredging ‘their’ sand.

The design approach focused on a simple website that addresses the needs of both professionals and locals. Claire and Simon decided that the design should answer core questions and myths, stick to plain English, use local materials and represent risks  without alarming local people.

Is it Ethnography?
After the presentation the audience participated in a lively debate, which had a particular focus on the true meaning of ethnography. For some designers ethnography was a new concept and their reactions during the Q&A sessions and after the presentations indicated that they found both Simon and Miles’ presentations truly thought provoking. Some felt that rapid ethnography with a specific structure and design agenda was different from “ethnography” and needed a new term associated to it.

Theoretical research has two main aims – the validation of existing knowledge and the acquisition of new knowledge. Flow uses research to acquire and validate specific knowledge, the context in which services and products are used.  Flow uses principles and techniques taken from social sciences such as sociology, anthropology and psychology to inform design decisions. Our main aim is to design solutions that work outside of design studios, laboratories and meeting rooms. As a result, we often use appropriate research techniques to focus on specifically targeted contexts and activities. A term that is often used to describe this work is Design Ethnography.

Simon's presentation

_____________
Many thanks to Danny Hope and former Flow Consultant Harry Brignull for organising the event.

--Ofer Deshe