THE 50PLUS BEST PRACTICES SERIES

Conducting Website Usability Research^©

Volume 2

Table of Contents

• Introduction
• Qualitative vs. Quantitative Interviews
• Assessing Site Usability
• Leveraging other Sites
• Assessing Site Imagery

Introduction

At the beginning of the 21st century, website usability research was probably the fastest growing area within the market research industry. Companies were finally taking their internet site seriously, everyone was pouring money into developing sophisticated sites, ecommerce was booming, new site concepts were being developed on a daily basis. Then the market crashed, budgets shrank, and the resources available for developing and testing new sites dried up.

But research moves in cycles. Companies have begun to remember the importance of evolving, expanding, and refreshing their web presence. And as web development dollars have begun to grow again, so have the resources for website research.

But, while it is clear that companies are returning to conducting usability research, the jury is still out on how usability research should be conducted. This whitepaper is meant to provide professional researchers and clients with a perspective on conducting usability studies. It does not attempt to say what the “best” way is, since the research process should be shaped by the information you need and the decisions that must be made. Rather, it is meant to provide a point of view and some suggestions that may help shape the discussion.

↑

Qualitative vs Quantitative Interviews

Should you ask why or ask how many?

At the simplest level, there are two basic approaches to Website Usability Testing and each has its own strengths and weaknesses. In this document, we’ll focus primarily on Qualitative Usability Interviews (see Volume 3 of the 50PLUS Best Practices: Managing Online-Surveys for tips on creating an effective and efficient online survey). In the sections below, I have outlined when each approach is generally most appropriate

Qualitative Usability Interviews

Qualitative website usability interviews are traditionally done with one respondent, or “site visitor”, and one interviewer sitting together at a computer. The interviewer prompts the respondent with a series of open-ended questions as he directs his attention to different areas of the site (e.g., home page, nav bars, product layout) and asks him to attempt a variety of tasks (e.g., sign up for an email newsletter).

By watching how respondents navigate the site and by listening to their comments, patterns emerge in how visitors respond to both the site as a whole and to specific elements. As the interviews progress, the moderator is able to drill down deeper and deeper into these areas. Research questions that can best be addressed by this method include:

• What is the overall impression of the site?
• Is the site easy to navigate and does it follow a logical design?
• What are the obstacles to completing important site tasks?

Because the interviews are qualitative, a large number of interviews are not necessary. We typically recommend 10 – 15 interviews per user segment. Therefore, if you are just testing one segment consider doing about 12 interviews, but if you’re testing two segments (e.g., Design Engineers and Line of Business Managers) try a total of 18 - 20 interviews.

Quantitative Usability Surveys

Quantitative surveys are great when you want to test alternative design elements or choose between different site layouts, or select navigation iconography. But, quantitative surveys are generally not very helpful in understanding the visitor’s experience.

Use a quantitative approach when you need a statistically reliable answer on which design alternative (e.g., page layout, navigation, iconography, etc) visitors will prefer. Examples of the types of questions you can answer best with a quantitative approach include:

• Do visitors prefer lots of small product pictures on one page – or a few larger pictures on two pages?
• Which icon style most clearly communicates its meaning?
• What percentage of respondents can successfully find a specific item from the product inventory?
• Which navigation strategy results in the fewest dropouts?

↑

Assessing Site Usability

Don’t Count Clicks.

Companies often believe that the fewer the clicks, the more usable the site. Nothing could be further from the truth. Visitors will report a much more positive experience for a page that uses five clicks that are logically designed and easy to find than for one that only takes 2 clicks, but requires them to hunt through a page of content to find the right link.

Try to resist the urge to focus on the number of clicks it takes to find a specific page or how many seconds it takes to complete a task. Instead, investigate the respondent’s experience finding the material or completing the task.

• What did you expect when you clicked on that link?
• How did you decide where to click?
• Did the path feel logical, fast, easy to find?

The answers to these questions will be much more useful than just learning that it took 3 clicks to find the account sign-in page.

↑

Leveraging other Sites

Learn from others.

Research participants cannot design a web page for you. If you ask a respondent “How would you like to see the product page laid out?” they will rarely provide a useful answer. After all, they are web surfers not web designers.

At the same time, respondents are excellent at telling you what they like, or don’t like, about a web page. That’s why it is often a good idea to visit other sites during the usability interview. If you’re not sure which approach to take when designing the sign-in page – take the respondent to 2 or three other sites and have them compare the different approaches. That way you can quickly discern some of the advantages and disadvantages that go with any design decision.

The goal is not to “rip off” someone else’s design; you are just giving the respondent something to look at and respond to. This will stimulate the discussion and provide more actionable insights than by asking them to come up with the design on their own.

Similarly, it may be useful to have them visit some of your competitor sites. This is an opportunity to understand how your site “stacks up”.

↑

Assessing Site Imagery

Your site is your brand.

It is very easy to become so focused on the mechanics of a site that you forget that your site is also an expression of your brand. In addition to the more functional or task-driven research activities, have the respondent take a step back and respond to questions like:

• What is the first impression you get from the home page?
• What kind of person is this site designed for?
• How does the site fit with your perceptions of the company?
• How does the site make you feel about the company?
• What is the company trying to say about themselves?

These questions are most effective early in the interview before the respondent has become “too educated” about the site and the company. Their initial reactions to the site will give you the best indicator of the “image” the site is communicating.

↑

»View the PDF file of this document.

Contents are the property of:
50PLUS MARKET RESEARCH, 2090 Vistazo East, Tiburon CA 94920
(415)-366-7270 Phone
(415) 789-0366 Fax
jacob@50plusresearch.com