Tips for Online Surveys
By Arthur Prokosch

Arthur Prokosch
Arthur Prokosch
Online surveys have emerged as an efficient way for nonprofits to gather data from their constituents, donors, and even their peers, but success depends on what you’re seeking, how you relate to your target audience, and how well you execute the details.

Here are tips to ensure you get the most and best data possible.

Plan Your Survey

Be as clear and specific as possible about what it is that you’re looking for. Write it down. Run it by coworkers. As you create and test the survey, refer back to this purpose to make sure the questions you ask will give you data that meets it. If you don’t, you risk creating vague questions and meandering surveys that generate useless data.

Be realistic about how motivated and available your intended survey-takers are. If you’re not sure, take the time to run a test with 10% of your list before investing more resources.

If your audience is motivated, providing an incentive for completion could be optional. Otherwise, be prepared to offer an incentive, e.g., a white paper download, or a raffle entry for a larger prize. While responses from only the most motivated constituents can be informative, you want to hear from as wide a spectrum of your audience as possible.

If your survey software allows, break up your survey into multiple pages if it’s longer than two to three screens. Be sure to indicate “page 1 of N” at the top, so that your respondents won’t get intimidated and give up, and can set aside an appropriate chunk of time. This has the added benefit of saving the responses on any completed pages, even if respondents do give up partway through. Also be conscious of length: if you go beyond three pages of this size, you’ll get fewer responses, many of which will be incomplete

Closed and Open-ended Questions

To be sure the questions you ask will give you the data you want, you need to ask them in the right format. The goal is 1) to get your respondents to answer on topic, and 2) to collect answers in a way that allows as much summarization and analysis as possible.

A closed question type is one where the respondent clicks an answer visible on their screen. Because it’s quicker and easier both for the respondents to answer these questions, and for the preparer to graph, total, and manipulate the results, closed questions get used more often than open-ended questions.

Note: For multiple-choice questions, be sure to include options like "don't know," "N/A," and/or "other" as appropriate.

Multiple choice/choose one is the most restrictive question type, but that makes it the easiest to analyze. Multiple choice/choose many is a good alternative when you’re looking for a full spread of responses, or when it becomes obvious that a “choose one” question won’t elicit accurate answers. The downside is you can’t compare the answers as easily: they won’t sum to 100%, and generally it will be more difficult to make comparisons between answer choices (because “both” will have been possible). These generally appear as checkboxes on the survey.

Open-ended questions are questions that permit an unconstrained response. They generally allow richer detail in areas that you couldn’t have anticipated, and can convey to the respondent that their true opinions are being recorded and heard. They’re more difficult to process, however: you can’t total the results or graph them; the most you can do is painstakingly assign responses to subjective categories, or else cherry-pick a few to quote in a written report.

Pulling It All Together

Write your survey on paper first. Do not design your survey within the web interface of your survey software. Revisions and reordering will be frustratingly slow and awkward. Only when it is near ready to go, and you have at least verified what question types are best to use, should you transfer the questions into the online survey framework.

Whenever possible, arrange a test run of your survey with three to six people who are similar to your respondent pool. Arrange to talk with them afterwards, and make sure they do most of the talking.

Start early. The surest way to end up with embarrassing typos, ineffective questions, or broken invitation links is to be rushed. Starting early allows you to review and test your survey.

Test the email message. This includes the subject line, text, and links. Make sure that a link to the survey itself shows in the first screen of email text.

Be careful about sending reminder emails. If your survey software has an option to send reminder emails only to people who haven’t responded yet, use it. In any case, don’t send more than two reminders, or you’ll alienate your constituents.

On the final page, next to any demographic questions, give respondents the opportunity to get more involved in your organization, for example, with a checkbox allowing them to subscribe to an e-newsletter.

Most modern systems, such as SurveyMonkey or Zoomerang, have additional useful features, including templates, tools for publicizing your surveys, or the ability to filter and search though the results. For recommendations on which online survey system to choose, see TechSoup and Wikipedia have good lists of software and features.

Arthur Prokosch, Online Services Manager at Third Sector New England, creates, troubleshoots, and analyzes online surveys.