How to avoid bias in your questionnaires?

A well-constructed questionnaire is one that will enable you to gather the most authentic, reliable, and realistic responses from your respondents. It is therefore extremely important to eliminate any risk of biasing their responses.

💡 A bias is an element that systematically distorts the authenticity of the interaction (direct or indirect) between the respondent and the person seeking information.

Here are some simple and effective ways to avoid bias in your surveys:

• Avoid yes/no questions
• Do not ask leading questions
• Pay attention to the order of questions and responses
• Prioritize anonymity whenever possible
• Clarify your language
• Eliminate assumptions
• Sometimes prefer open-ended questions

‍

Avoid yes/no questions

Try to avoid questions that can be answered with a simple yes or no. Indeed, respondents tend to want to say yes – to appear positive and ambitious. This is known as acquiescence bias.

Rephrase your questions. For example, for concept testing, instead of "Do you find it essential to have attractive packaging when buying cookies?", ask: "Among the following elements, which do you find essential when buying cookies?".

Do not ask leading questions

Be careful with leading questions, which encourage respondents to answer in a way you desire. This sometimes results in confirmation bias. You can remedy this by avoiding unnecessary adjectives in your questionnaire. For example, do not say "What do you think of [brand]'s delicious chocolate?". Instead, ask "What do you think of [brand]'s chocolate?".

Also, remember that responses to scale questions can also be leading. Ensure that your scale question responses include an equal number of positive and negative options, and critically, not solely positive options.

Pay attention to the order

The way you order your questions and responses affects how people react. This is known as order bias.

In questions

Order bias can occur when a question you have asked – or an item you have presented – impacts how your respondents perceive the rest of the survey. For example, if you present a series of advertisements, all subsequent ads will likely be compared to the preceding ones in the respondent's mind, even unconsciously. Here's what you can do to avoid order bias in questions:

• Do not provide answers to upcoming questions within earlier ones. Consider the order in which you ask questions to ensure you progress from the most general at the beginning to the most specific towards the end of the survey.
• Whenever possible, randomize the questions. The Episto platform integrates this feature, allowing you to shuffle the order of questions in your questionnaire.

In responses

The given responses can also be influenced by the order in which you present them. People are more likely to choose the response(s) located at the top of the list. If this occurs, you will not have an accurate representation of your audience's opinions. Episto's programming platform allows you to randomize response options. It is advisable to do this for all responses to all questions, except when using scale responses.

Prioritize anonymity whenever possible

Do not reveal your brand identity! If you do, you risk causing a sponsorship bias, meaning respondents will feel compelled to give positive answers about the brand asking the questions. Furthermore, it prevents respondents from letting their perceptions of your brand cloud their answers.

You generally want to know what consumers think of your brand, but sometimes you want to know what people think independently of your brand image. For example, if you are conducting new product development research, you want to understand consumer opinions without your brand's image – whether positive or negative – influencing the responses.

During social media recruitment campaigns, Episto systematically ensures that the brand is not cited or mentioned to avoid sponsorship bias.

Clarify your language

Avoid confusing respondents or leaving room for interpretation in answers by objectifying your language. For example, instead of using terms like "recently," "often," or "rarely," specify the time period, such as "daily," "weekly," or "once a year." Ensure you understand how your ideal customer communicates, which will help you adapt your language.

There are 6 levels of French language proficiency, from A1 (beginner level) to C2 (perfectly correct expression on all subjects). Clear language corresponds to level B1, which is understood and mastered by 95% of the population. It is therefore advisable to use this language level in your questionnaires to be understood by all your respondents. Also, remember to limit Anglicisms and explain any technical terms you need to use in your questionnaire.

‍

Eliminate assumptions

Avoid making assumptions that force respondents to answer untruthfully. You can easily remedy this by asking yourself whether or not to include "None," "Not applicable," and "Other" options in your list of responses. For example, the question "What type of laptop do you own?" assumes the respondent owns a laptop; therefore, you must ensure they can answer honestly by adding "None" or "Not applicable" options.

Instead of "What did you like about your last vacation?", rephrase the question into two distinct parts: "What did you think of your last vacation?" with options like "Very pleasant," "Quite pleasant," etc. Remember to add an option for the answer "I did not take a vacation [in the past year]," then follow this with an open-ended question like "Why do you think that?". With Episto, it is even possible to customize the wording of a question based on the answer given to the previous question.

Sometimes prefer open-ended questions

Favor open-ended questions if you are unsure you can provide an exhaustive list of responses. In this case, to avoid bias, it is better to analyze written responses rather than forcing answers into an exclusive choice.

For example, when proposing brand names from a defined sector, it is not complicated to provide respondents with a simple, exhaustive list of brand names. However, when it comes to reasons why respondents wear makeup or go on vacation, some answers may be unexpected – but no less interesting!

Thanks to the intelligent features of the Episto platform, it will be easy for you to analyze these open-ended responses and account for unexpected answers.

Do you want to build a questionnaire adapted for social media studies that truly engages your respondents? Check out this article to discover Episto's tips.

‍