Flow helped the Training and Development Agency to build a user experience roadmap for their website and create a user experience scorecard for measuring its success.

Ethnographic approaches, such as contextual enquiry and experience labs, help organisations understand their target customers needs, behaviours and motivations. To get really insightful discoveries, researchers immerse themselves as deeply as possible in the lives or jobs of a limited sample of target users and try to discover what those people really think and need.

On the other end of the spectrum lies automated quantitative usability testing. Here you never see your users at all. But you get accurate measurements of how successful they are at achieving key tasks on your site. Gaining a quantitative understanding of your website's performance lets you chart your site's improvement over time, and identify where it could be performing better.

Ethnographic techniques tell you what your users want to do. Quantitative testing tells you how many of them are managing to do it.

And often, successful UX strategies will combine the two. Our project for the TDA did.

Building a UX strategy for the TDA

We began by conducting contextual research with people from the TDA's 7 target user groups - including teachers, support staff and school leaders. We gathered stories of actual experiences that they had lived through, and the sequences of activities they had engaged in. We also played some simple participatory design games with them to bring out ideas for the "ideal" TDA website.

With a separate sample, we usability tested the existing website. Using a huge quantity of sticky notes, the researchers put all the data together and analysed it. They uncovered a selection of 70-80 tasks that the different target groups needed to perform on the website. (We also discovered that there were really only 4 groups with different needs, rather than 7).

Next came a "task matrix". For each task, we identified:

• The audiences who do it
• Related tasks or sub tasks
• Triggers that cause someone to engage in the task at a certain point (events like assessments, deadlines, changes in staff)
• Target web pages where useful information would be found.

Straight away, this let us see where there were improvements to be made. If we struggled to find suitable target pages, or if the information or findability of a target page seemed poor, we knew we had an opportunity to improve things.

To take us beyond expert opinion, we also used the TDA's web analytics data. We checked how many people were visiting each target page, and assessed that figure in relation to task importance and audience group size. That gave us additional evidence about whether pages we easy to find or not.

Finally, we worked with stakeholders to assess the effort needed to improve the website for each task. Plotting each task on a chart of priority versus effort (for users and for the organisation) gave us a solid roadmap: small, high-value changes first; larger changes later.

User experience scorecard

To make it easier to measure and chart improvement in the site's UX, we set up a user experience scorecard using UserZoom.

UserZoom performs automated remote usability testing. It asks a quantitative sample of target users to try doing certain tasks, gathers clickstream data and quizzes them to check comprehension and attitude. The scorecard treats user tasks as key performance indicators of the site's user experience. The TDA is running benchmark UserZoom studies every six months, and the results are being fed onto an interactive dashboard.

The scorecard will help the TDA team to monitor the impact on the user experience as they work through the steps on their UX roadmap.

Flow's team: Mary Henley, Anthony Mace, Claire Mitchell, Leisa Reichelt, Sarah Herman, Pav Chahal, Nick Bowmast, Vanessa Kirby, conducted the research. Karen Wall looked after the client relationship.