THE 50PLUS BEST PRACTICES SERIES

Managing Online Surveys©

Volume 3

Table of Contents

Introduction

The impact of online survey technology on the market research world is an old story. Everyone knows the value and benefits of online quantitative research.

But while the online survey technology looks seductively simple, many have found the management of online research surprisingly difficult. This is especially true in B2B research where sample costs are high and finding the right respondents can be tremendously difficult.

This Best Practices whitepaper is meant to provide professional researchers, or clients conducting research on their own, with a few “tips” that can help them avoid some common online research problems and improve the efficiency and effectiveness of their current research processes.

↑

The Value of a Pre-test

An ounce of prevention

Surprisingly, the majority of online surveys are sent into the field without a pre-test. While researchers know that a pre-test is a good idea, they often succumb to client pressures and the fear that there is:

  • No time in the schedule
  • No money in the budget
  • No need

In reality, no one has ever regretted running a pre-test. One extra day and a few hundred dollars will save you thousands of dollars and weeks of wasted effort. A pre-test is a hundred times better than sending the link out to your colleagues or clients asking them to help “check the survey”.

The key benefits of the pre-test include:

  • Flag any questions that are causing a high drop-out rate
  • Alert you to programming errors
  • Provide an accurate estimate of your final sample requirements

In some cases, you may only have a limited number of names, and don’t want to “waste” them in a pre-test. In these cases, simply buy some inexpensive names from your supplier. While you can’t use their data, they’ll work perfectly to test the survey programming.

↑

Monitoring Data Quality

A few numbers that will save your life

Any good online survey provider, or survey software, will have a comprehensive set of online reporting tools. These tools give you real-time access to important metrics that serve as an early warning system for problems that can ruin the best data collection plan.

Click Through Rate

Track the percentage of people who open your survey invitation. If the “click through rate” is lower than your expectations (or the norms your online provider typically achieves), it means you have to re-work the invitation. Some of the first changes you might try are:

  • Describe the survey topic in a more interesting way
  • Shorten the copy in the invitation
  • Raise the incentive

Drop Outs

Look for questions that have a high drop out rate. A single question can dramatically reduce your number of completed questionnaires and significantly raise your costs. Ways to make these questions less demanding include:

  • Delete some features or attributes
  • Split the question into two parts

Total Completion Rate

If your total completion rate is low, but no individual question has a very high dropout rate, it may mean that the overall survey is too long. Cut out a few questions or consider removing a few variables from any large grids.

↑

Constructing an Effective Invitation

Clarity – but with a little sizzle

The invitation to take your survey is one of the key factors in determining your participation rate. Small changes in the text can result in big changes in participation rates. And, moving from a 4% to a 5% participation rate increases the number of respondents by 25%.

Some guidelines to remember in writing an invitation:

  • Make it short and to the point – no one likes reading long emails
  • Avoid marketing hype – explain don’t sell
  • Include the topic of the survey
  • Craft the topic in a way that your target will find interesting – an interesting topic makes for motivated respondents
  • Create a custom invitation for each target segment – so that the invitation can speak directly to the needs and interest of each segment
  • Include the company sponsoring the survey if possible - this increases credibility and will result in significantly higher participation rates
  • Clearly spell out the amount of the incentive

↑

Using an Incentive

A few dollars go a long way

The desire to save a few dollars often leads companies to run the survey without an incentive. This is almost always counter productive. Even a small incentive substantially lifts your participation and yields higher completion rates. This helps insure that you have high quality participants who value their own time.

Sometimes clients will be afraid that they are “Bribing” the respondents. The incentive is not a bribe. Incentives:

  • Provide the respondent with a reason to participate – just like a sales promotion provides the target with a reason to buy.
  • Show a level of respect for the respondent. You are willing to compensate them for their time.

Incentives are typically very small expenses within the total cost of the project. We often use a drawing for one of five $200 gift certificates on Amazon.com.

↑

Leveraging Online Technology

Don’t shoot for the lowest common denominator

One of the great things about online surveys is the technology that you can embed within the survey itself. You can show movies, demonstrate products, or explore a competitor’s website.

At the same time, it is important to take the respondents’ technological sophistication and computer systems into account. Often, out of fear, researchers like to “dumb down” the survey so that every web browser and dial-up user can see it. But that is typically an over-reaction. Imagine your core target, and use a level of technology that is appropriate to that target. True, you may lose a few technology laggards, but you’ll more than make up for it by having a dynamic survey that engages your participants and results in higher completion rates.

At the same time, recognize that the more technology you pack into the survey, the greater the number of respondents who will have some technology problem. Completing your survey is not your respondent’s first priority. It is unreasonable to expect a respondent to go to the trouble of loading the latest Flash Player or re-setting their monitor’s resolution in the middle of a survey. They’re much more likely to just go back to reading their email.

↑

Research not Spam

Part of the problem or part of the solution?

Just as unlisted numbers and hang-up rates have changed the economics of telephone surveys, spam poses a danger to the legitimacy of online research.

In today’s spam filled world – every researcher wonders:

  • Will my online survey be dismissed as spam?
  • Am I contributing to the level of spam by doing online research?

The best way to fight spam is to make sure that you are responsible in your use of email lists. The two primary rules are:

  1. Don’t harass your list. Send the invitation and then one follow-up email. Bombarding the list with repeated pleas to take your survey only generates angry responses.
  2. Use good lists. Make sure your lists are opted-in. And if you’re renting a list, try to find providers who conform to the standards of the Council of American Survey Research Organizations (CASRO).

↑

»View the PDF file of this document.
Copyright © 2007, 50Plus Market Research, All Rights Reserved