With so many different types of surveys and questionnaires, covering a wide range of topics and conducted for a multitude of reasons it is difficult to devise the definitive guide to designing a survey.

However, whether the survey is to gauge customer or employee satisfaction, whether it is being used for market research or even marketing research, has an academic aspect or is part of a medical study or simply being used to sample public opinion there are some design considerations that are common to them all.

Objectives

Right from the start, for any type of survey, the survey's goals and objectives must be established.

A survey's Raison d'être must be its objectives, that in turn will help define the questions to be asked, and the manner that the respondent is required to respond.

However, just establishing the objectives is not enough, it is important to use the objectives to challenge every question and asked, 'Does this question contribute towards achieving the survey's stated objectives?'.

Clearly state the survey's objectives and continually refer back to them to ensure that the survey does not at any time and in any way deviate from achieving the objectives.

Can the respondent answer truthfully?

This might seem a strange consideration but it is common to see surveys containing questions where certain respondents, through no fault of their own, or any reluctance on their part, are unable to answer truthfully.

Normally this is the result of the people who have designed the survey asking questions from their own perspective, experience and knowledge, or where the question has been mandatory but the answer options do not cater for all possible answers.

For example take a question like 'Please indicate what class you usually travel?', where the answer options are 'First', 'Business', 'Economy'.

Setting aside that carriers invent their own categories such as 'Premium' that would make it difficult for the respondent to determine if 'Premium' was more akin to 'Business' than 'Economy', more serious is if the question was made mandatory it does not cater for those that may not have travelled, in which case for them to continue with the survey they would have to select one answer option forcing them to not answer truthfully.

Check each question from a respondent's perspective, and ensure that they can answer truthfully.

Does the question contain subjective words?

Examples of subjective words are 'tall', 'small', 'regular', 'a lot', 'frequently', 'rarely'. They are words that mean different things to different people.

Take the question 'Do you regularly drink alcohol?'; some may consider every day to be regularly, others once a week, once a month, or with every meal.

The survey's publisher will have their own view as to what they mean by 'regularly' but unless they quantify it to the respondent, seriously compromised information will be gathered.

If subjective words are used, they must be quantified either in the question or in the answer options such as 'Regularly (At least once a day)'.

Check for subjective words and if found either change or quantify them.

Is the question bias?

The following are two example of question bias:

Do you agree with irresponsible drivers who say that speed cameras have no effect on reducing accidents?
How often do you attend church?


In the first example the word 'irresponsible' creates a bias against the group of people who say that speed cameras have no effect in reducing accidents.

In the second example the word 'church' causes several problems. 'Church' is specific to certain religions, there are other religions that do not recognize the word 'church' and may instead refer to their place of worship as 'Temple', 'Mosque' or 'Synagogue'.

The word 'church' therefore puts certain respondents in a dilemma, do they translate 'church' to mean their own place of worship, or do they answer the question pedantically? The survey publisher will have no way of knowing how the respondent chose to answer.

The use of the word 'church' will effectively define the whole survey as being from a specific viewpoint, probably causing respondents from minority groups to feel that the whole survey is bias towards a specific group of people.

In addition to there being bias in the phrasing of the question also consider if there is bias in the order that the questions are asked or the order that the answer options are displayed. Good online survey websites will allow the order that the answer options are displayed to be made random.

Check for bias, consider niche groups. For closed questions consider including an 'Other' answer option, or allowing the respondent to select a positive 'no answer' answer option.

Does the question ask only one question?

Consider if the question being asked is only asking one question; take for example the following:

Did you find the trainer engaging and knowledgeable?

If the respondent found the trainer engaging but not knowledgeable, or vice versa, this type of question puts the respondent in a dilemma.

If they were to answer 'No', it would suggest that the trainer was neither, if they answer 'Yes' then it would give the impression they were both.

The survey results will be seriously compromised, as there is no way to know which way each respondent was answering.

Check each question and ensure that you are only asking one question, where you are asking more than one split them into individual questions.

Does your question contain abbreviations and/or jargon?

There are some surveys, that are targeted at specific groups, where jargon and abbreviations are acceptable, for example a survey aimed at surgeons researching a medical procedure.

However, the rule is always the same, consider all the respondents and ensure that if jargon or abbreviations are to be included that they will be known to all, or the survey will include an explanation as to any jargon or abbreviations that has been used.

Only include abbreviations and/or jargon if you can guarantee that all the respondents will understand the language.

Test your survey.

The beauty of online surveys is that they can be fully tested not only from a respondent's point of view but also to ensure that the results can be analysed in a way that will allow the survey's objectives to be met.

Where possible enlist the help of colleagues as it is often difficult to properly test if you are the survey designer and the tester, the designer tends to test for exceptions they have already thought about, not those that they may have over looked.

Don't squander the opportunity for testing, keep in mind that an online survey allows you to test the survey from the perspective of the respondent and also to confirm that the analysis will achieve the survey's objectives.

Martin Day is a Director of Survey Galaxy Ltd a web site that allows anyone to create, design and publish online surveys. Please visit the website for more information on conducting online surveys and questionnaires.