SURVEY TIPS

Sample size

• Keep sample small
  
• Can tell respondents that they are part of a select group (appeals to scarcity)
  
• If using a sample of the population, you can administer multiple surveys, using different people, so that folks don't get tired of being surveyed.

How many times do you contact folks?

• Send at least 3 reminders
  
• Change the day of the week and time of day that you send reminders, because students and professors tend to have times that they are more likely to respond.
  
• Be sure to remove those who have already responded from your reminder list.

Use a mixed-method of contact

• Use of both paper and email contacts has been found to be effective. For example, send a letter from the provost before the survey.
  
• Follow-up with a postcard
  
• A study has shown that sending a post-card before the survey has increased response rate by 9% (Public Opinion Quarterly)

Structure of email

• Subject line
  
  • Don't use the word "survey"
    
  • Mention EMU to make your email stand out from SPAM
    
  • On the final reminder, one study found that having the subject line blank got the most responses.
    
• Body of email
  
  • Keep it short and use text only (not html because some SPAM filters look for html)
    
  • Put the hyperlink near the top of the email.
    
  • Include contact information out of courtesy.
    
• Techniques
  
  • Tell respondents that they are part of a select group.
    
  • Give a deadline; limited amount of time to complete.
    
  • Ask for their help (i.e., appeal to their helping tendencies).
    
  • Tell them to let you know if they don't want to be contacted again.
    
  • Give a rough estimate of completion time.
    
  • Promise confidentiality, if you can. But, don't overemphasize it. Overstating confidentiality can decrease response rates and leads to suspiciousness.
    
  • Make sure that you can defend the content of your email to the Provost and President.
    
• Things that typically cause emails to go into 'Junk Mail' as SPAM
  
  • Capital letters
    
  • "from" name contains underlines or numbers (e.g., sally_52)
    
  • A large number of blank lines or spaces in the body of the text
    
  • "Click here" or "Click below"; it is better to use the word "submit"

Response Rate

• Rule of thumb - at least 50%
  
• If your response rate is low, check to see how the demographics of your sample compare to the population.
  
• Send follow-up emails not more than 1 week apart.
  
• After the first 2-3 days, the response rate typically tends to drop off.
  
• Once the response rate drops off, send another follow-up email.

Web Design

• Appearance
  
  • Have a short introduction at the top
    
  • Avoid drop-down boxes (they are easy to miss); use radio buttons or check boxes instead.
    
  • Make sure that the layout is vertical and that respondents don't have to scroll horizontally.
    
  • Use plenty of white space and headings.
    
  • Number your questions so that people don't get lost.
    
  • Use yes/no instead of having them check all that apply (will be easier to use the data and forces respondents to read all of the options).
    
    Example:

		Yes	No
	Library?
	Athletic Facilities?
	Dining Hall?

• Length
  
  • Don't ask demographic information if you already have it.
    
  • If it is long use multiple pages (downloads quicker).
    
  • Don't force responses.
    
  • Keep the survey under 20-30 minutes to complete.
    
• Salience of the survey topic
  
  • This is one of the best predictors of survey response.
    
  • Put interesting questions near the top and boring questions near the end.
    
• Open-ended questions
  
  • Use of "other" often indicates that you have not formulated your responses well.
    
  • Keep short, for your own sake.
    
• End of survey
  
  • Make sure the "submit" button is visible.
    
  • Have a "thank you" page after the submit button.
    
    • Lets them know the survey is over.
      
    • Can also use it to direct them to another website, or collect another survey.
      
• Timeline
  
  • Start early, at least a month before you want to the survey to go out.
    
  • Developing a survey is a long process and there are often many refinements along the way.
    
  • Allow time for pilot testing.
    
  • Take feedback seriously.

Guidelines and Request for Use of Web-based Tool form may be found at the Institutional Research and Effectiveness Office webpage at www.emu.edu/ir/ and click on the links to the web survey tool.

Resources

• Don Dillman is the guru in this area and has published studies on web surveys.
  
• Public Opinion Quarterly - www.poq.oupjournals.org (Hartzler Library has 1983 - present.)
  
• Web survey methodology website - www.websm.org
  
• Social Science Computer Review - ssc.sagepub.com (Hartzler Library does not subscribe.)
  
• Groves et al. (1992) "Understanding the decision to participate in a survey." Public Opinion Quarterly, 56:475-495.

Modified from Steven Porter's presentation for an Academic Impressions webinar on February 15, 2005.

Click here to view a copy of the form to fill out and return to the Institutional Research and Effectiveness Office to request to do a web survey.

Back to IR Homepage