10 Tips for Online Survey Research

April 26, 2010Sparxoo


Not long ago, market researchers depended on phone surveys and mall intercepts.  The Internet has made research more affordable, accessible, and timely.  In fact, most online research surveys can go from setup to response to data in a week or less.  We share tips for online research that will yield bigger impact.

1. Screen-Out Unwanted Respondents. There are millions of research respondents who are part of online panels, and not all of them have opinions that will be helpful to you.  Be sure to hone in on whose opinion matters, and screen out the rest.  A simple market segmentation will help you think about who to include and exclude.

2.  Customer vs. Non-Customer. The most common mistake I have seen in research is the focus on only customers.  Typically, organizations already know the most about their own customers.  It’s easy to talk to your own customers, and there should already be an ongoing dialogue.  If you are focused on acquisition and growth, then the biggest knowledge gap is typically about non-customers.  If this is the case, be sure to have strong representation from non-customers whom you have prioritized as part of your growth initiatives.

3. Strike the Right Balance Between Time and Reward. Have you ever heard of respondent fatigue syndrome?  Basically, the more questions that are asked, the less focus the respondent has, and the more likely they are to start choosing any answer.  If you are not paying your respondents, or are offering a small bounty, then keep the survey short (under 10 minutes).  If you are offering a significant reward for completing the online survey, then hopefully your respondent will be more motivated.

4. Build On What You Know. There is already so much information available online for free that can help you build a base level of intelligence.  You’ll want every question to count, so don’t repeat questions that you know the answer to or can easily get the answer to.  For example, you can easily find out the latest trends in online advertising and e-commerce, so skip the basics.

5. Keep “Open-Answered” Questions to a Minimum. By the time you commence an online research study, you should have a thorough understanding of the issues at hand.  If you are asking lots of “open-answered” questions, that means that you don’t know the range of potential answers.  If this is the case, you should really be doing qualitative research such as depth interviews or focus groups first.  “Open-Answered” questions are not ideal for an online research study because they are difficult to analyze and sift through.

6.  Force Choices. It might be easy to ask “Do you like feature A?”  “How about B?” and “How about C?” and then compare the yes and no responses.  However, this will give you some false positives and won’t give you a true prioritization.  Consider the situation where most of your sample likes A, B, and C.  The conclusion would be that you should greenlight features A, B, and C.  However, if you are to ask which feature is most important, it’s possible that the vast majority choose C.  The new conclusion would be to prioritize feature C.  Try to structure questions to force choices and identify true preferences.

7. Randomize Lists. Assuming you go with our advice in #6, you will likely have questions which offer multiple choices.  Answer choice order can influence respondent selection.  For example, if you have a long list, you might see that more of the earlier choices are selected relative to choices at the end of the list.  Most online survey software provides an option to randomize your list to avoid bias in the answers.

8.  Be Careful of Pricing Questions. While most online market research responses provide good direction, pricing is often less reliable.  If you are going to pursue pricing questions, employ more advanced techniques to simulate real world decision-making.  You’ll want to get responses to enough different answers that you can start to draw a demand curve to get a sense of price elasticity.  In general, beware of drawing too many conclusions from price testing in online surveys.  There are other ways to get at price testing online such as doing A / B splits on commerce offers online, or even to test via Google Ad Words.

9.  Consider Sample Size. Increasing your responses from 250 to 1,000 will approximately double your precision.  If you plan for advanced analysis in which you subsegment respondents, be sure to increase sample size.  I have worked on online surveys ranging from sample sizes of 250 to 4,000 based on the complexity of the planned analysis.  You also may want to set quotas for certain types of respondents (i.e. recruit users of a competitor product in proportion to their actual market share).

10.  Keep It Affordable. For more advanced panels, you can access a panel (74 research panels provided by GreenBook.org).  You can also use word-of-mouth and incentives with your own customers to get respondents inexpensively.  Either way, consider using a free or inexpensive research survey tool such as SurveyMonkey or Zoomerang to keep costs at a minimum (note that Zoomerang has a panel of over 2 million respondents that they can link up to your survey).

Image by ilker . from Stock.Xchng


Learn more about Market Research.