Tips for Online SurveysBy Arthur Prokosch
Here are tips to ensure you get the most and best data possible.
Plan Your Survey
Be as clear and specific as possible about what it is that youre looking for. Write it down. Run it by coworkers. As you create and test the survey, refer back to this purpose to make sure the questions you ask will give you data that meets it. If you dont, you risk creating vague questions and meandering surveys that generate useless data.
Be realistic about how motivated and available your intended survey-takers are. If youre not sure, take the time to run a test with 10% of your list before investing more resources.
If your audience is motivated, providing an incentive for completion could be optional. Otherwise, be prepared to offer an incentive, e.g., a white paper download, or a raffle entry for a larger prize. While responses from only the most motivated constituents can be informative, you want to hear from as wide a spectrum of your audience as possible.
If your survey software allows, break up your survey into multiple pages if its longer than two to three screens. Be sure to indicate page 1 of N” at the top, so that your respondents wont get intimidated and give up, and can set aside an appropriate chunk of time. This has the added benefit of saving the responses on any completed pages, even if respondents do give up partway through. Also be conscious of length: if you go beyond three pages of this size, youll get fewer responses, many of which will be incomplete
Closed and Open-ended Questions
To be sure the questions you ask will give you the data you want, you need to ask them in the right format. The goal is 1) to get your respondents to answer on topic, and 2) to collect answers in a way that allows as much summarization and analysis as possible.
A closed question type is one where the respondent clicks an answer visible on their screen. Because its quicker and easier both for the respondents to answer these questions, and for the preparer to graph, total, and manipulate the results, closed questions get used more often than open-ended questions.
Note: For multiple-choice questions, be sure to include options like "don't know," "N/A," and/or "other" as appropriate.
Multiple choice/choose one is the most restrictive question type, but that makes it the easiest to analyze. Multiple choice/choose many is a good alternative when youre looking for a full spread of responses, or when it becomes obvious that a choose one” question wont elicit accurate answers. The downside is you cant compare the answers as easily: they wont sum to 100%, and generally it will be more difficult to make comparisons between answer choices (because both” will have been possible). These generally appear as checkboxes on the survey.
Open-ended questions are questions that permit an unconstrained response. They generally allow richer detail in areas that you couldnt have anticipated, and can convey to the respondent that their true opinions are being recorded and heard. Theyre more difficult to process, however: you cant total the results or graph them; the most you can do is painstakingly assign responses to subjective categories, or else cherry-pick a few to quote in a written report.
Pulling It All Together
Write your survey on paper first. Do not design your survey within the web interface of your survey software. Revisions and reordering will be frustratingly slow and awkward. Only when it is near ready to go, and you have at least verified what question types are best to use, should you transfer the questions into the online survey framework.
Whenever possible, arrange a test run of your survey with three to six people who are similar to your respondent pool. Arrange to talk with them afterwards, and make sure they do most of the talking.
Start early. The surest way to end up with embarrassing typos, ineffective questions, or broken invitation links is to be rushed. Starting early allows you to review and test your survey.
Test the email message. This includes the subject line, text, and links. Make sure that a link to the survey itself shows in the first screen of email text.
Be careful about sending reminder emails. If your survey software has an option to send reminder emails only to people who havent responded yet, use it. In any case, dont send more than two reminders, or youll alienate your constituents.
On the final page, next to any demographic questions, give respondents the opportunity to get more involved in your organization, for example, with a checkbox allowing them to subscribe to an e-newsletter.
Most modern systems, such as SurveyMonkey or Zoomerang, have additional useful features, including templates, tools for publicizing your surveys, or the ability to filter and search though the results. For recommendations on which online survey system to choose, see TechSoup and Wikipedia have good lists of software and features.
Arthur Prokosch, Online Services Manager at Third Sector New England, creates, troubleshoots, and analyzes online surveys.