- Keep sample small
- Can tell respondents that they are part of a select group (appeals to scarcity)
- If using a sample of the population, you can administer multiple surveys, using different people, so that folks don’t get tired of being surveyed.
How many times do you contact folks?
- Send at least 3 reminders
- Change the day of the week and time of day that you send reminders, because students and professors tend to have times that they are more likely to respond.
- Be sure to remove those who have already responded from your reminder list.
Use a mixed-method of contact
- Use of both paper and email contacts has been found to be effective. For example, send a letter from the provost before the survey.
- Follow-up with a postcard
- A study has shown that sending a post-card before the survey has increased response rate by 9% (Public Opinion Quarterly)
Structure of email
- Subject line
- Don’t use the word “survey”
- Mention EMU to make your email stand out from SPAM
- On the final reminder, one study found that having the subject line blank got the most responses.
- Body of email
- Keep it short and use text only (not html because some SPAM filters look for html)
- Put the hyperlink near the top of the email.
- Include contact information out of courtesy.
- Tell respondents that they are part of a select group.
- Give a deadline; limited amount of time to complete.
- Ask for their help (i.e., appeal to their helping tendencies).
- Tell them to let you know if they don’t want to be contacted again.
- Give a rough estimate of completion time.
- Promise confidentiality, if you can. But, don’t overemphasize it. Overstating confidentiality can decrease response rates and leads to suspiciousness.
- Make sure that you can defend the content of your email to the Provost and President.
- Things that typically cause emails to go into ‘Junk Mail’ as SPAM
- Capital letters
- “from” name contains underlines or numbers (e.g., sally_52)
- A large number of blank lines or spaces in the body of the text
- “Click here” or “Click below”; it is better to use the word “submit”
- Rule of thumb – at least 50%
- If your response rate is low, check to see how the demographics of your sample compare to the population.
- Send follow-up emails not more than 1 week apart.
- After the first 2-3 days, the response rate typically tends to drop off.
- Once the response rate drops off, send another follow-up email.
- Have a short introduction at the top
- Avoid drop-down boxes (they are easy to miss); use radio buttons or check boxes instead.
- Make sure that the layout is vertical and that respondents don’t have to scroll horizontally.
- Use plenty of white space and headings.
- Number your questions so that people don’t get lost.
- Use yes/no instead of having them check all that apply (will be easier to use the data and forces respondents to read all of the options).
- Don’t ask demographic information if you already have it.
- If it is long use multiple pages (downloads quicker).
- Don’t force responses.
- Keep the survey under 20-30 minutes to complete.
- Salience of the survey topic
- This is one of the best predictors of survey response.
- Put interesting questions near the top and boring questions near the end.
- Open-ended questions
- Use of “other” often indicates that you have not formulated your responses well.
- Keep short, for your own sake.
- End of survey
- Make sure the “submit” button is visible.
- Have a “thank you” page after the submit button.
- Lets them know the survey is over.
- Can also use it to direct them to another website, or collect another survey.
- Start early, at least a month before you want to the survey to go out.
- Developing a survey is a long process and there are often many refinements along the way.
- Allow time for pilot testing.
- Take feedback seriously.
- Don Dillman is the guru in this area and has published studies on web surveys.
- Public Opinion Quarterly (Hartzler Library has 1983 – present.)
- Web survey methodology website
- Social Science Computer Review (Hartzler Library does not subscribe.)
- Groves et al. (1992) “Understanding the decision to participate in a survey.” Public Opinion Quarterly, 56:475-495.
Modified from Steven Porter’s presentation for an Academic Impressions webinar on February 15, 2005.