10 tips for Creating Great User Research Surveys

Surveys are one of the most used tools in a UX researcher toolbox. It consists of a set of questions sent to a targeted audience. Also, surveys can be a quick, easy, and inexpensive method to collect users' attitudes and preferences. Essentially, data! Although if the survey is poorly designed, it won’t yield valuable insights!

Generally speaking, surveys are challenging to design. They are quantitative, impersonal, and give a lot of work analyzing, especially if you build them with too many open questions. That's why several user researchers only design and apply a few surveys in their projects.

Others choose to go around the web and pick one already designed by another user researcher, sometimes adapting it. However, those surveys are often inadequate and poorly written. Consequently, those pre-designed surveys cause biased results and no valuable data to create good insights.

On the other hand, in the UX research you are trying to get the deeper meaning behind the user's actions or motivations during user research. But with surveys, you are attempting to ask people to answer questions with a predetermined list of answers. So, during a survey, you can't expect to probe, dig, or get a more profound explanation from the user. Basically, if you ask for a reason why, you aren't always guaranteed to get one. And the level of description won't be as good as using an interview.

To overcome some of the challenges of using a user research survey, we decided to create some tips. The following tips were meant to help you, fellow UX researchers, design better surveys to give you accurate data.

‍

Start with the two major steps when creating a survey: plan and structure.

Before deciding which information to give and questions to create in the user research survey, start planning and then structure it. No one begins a house without the foundations!

‍

1. Planning the survey

Understanding the survey objectives like evaluating client satisfaction or evaluating workplace conditions is an essential first action. Secondly, you need to define the expectable consequences of having the survey, like improve client satisfaction or motivate change within a company. Third, recognizing potential user's issues will help you align on the priorities to research.

‍

2. Structuring the survey

This step is crucial for what's to come. The survey design! Which we will focus on today. One common way to structure a survey is to define the hierarchy of the information you want to present to the user and the issues you want to research: simple first, more complex near the end. And don't forget to control possible bias. Ensure the topics presented earlier don’t influence the answers of subjects that you want to appear later on the survey.

With these two steps completed, you can write the questions! But you need to take into account that not all questions are created equal. To help you with this challenge, we selected some dos and don'ts for the user research surveys.

‍

Now it’s time to apply the tips for creating better user research surveys

‍

Tip 1. Write an introduction to the participant

‍

At the beginning of the survey, writing an introduction to the participant will help him understand how he will contribute to the product/service and how much time he will need to complete it.

‍

Here is what should be in the introduction:

⦁ Give brief instructions on how to fill it

⦁ Tell him the approximate time to complete the survey

⦁ Explain the survey's purpose

⦁ Mention that will be anonymous

And at the end of the survey, don’t forget to thank him for his participation!

Tip 2. Ask one question at a time

‍

Bad example

How satisfied are you with the look and content of the site?

In this example, the participant may be confident with both. But what if he is only happy with the look?

‍

Another bad example

If you have a camera, what brand is it?

‍

Good examples

Do you have a camera?

If so, what brand is it?

‍

Tip 3. Be specific on the questions

‍

Bad example

Have you used the internet?

This question won't tell much. And if the participant only used it once? He might be confused about how to answer the question.

‍

Good example

How frequently do you use the internet?

▢ Diary

▢ Twice a week

▢ Once a month

▢ Never

You need to express frequency and quantity with specific number questions since words like frequently or rarely are too relative.

‍

Tip 4. Careful with ambiguity on the options

‍

Bad example

How many years have you worked at the company?

▢ 1 to 5 years

▢ 5 to 10 years

▢ 10 to 15 years

In this example, the participant will have difficulties deciding if he works at the company for 5 years.

‍

Good example

How many years have you worked at the company?

▢ 1 to 5 years

▢ 6 to 10 years

▢ 11 to 15 years

‍

Tip 5. Include options like 'Other', 'Not applicable' or 'I don't know'

‍

Bad example

What is your favorite type of cuisine?

▢ Italian

▢ Japanese

▢ Chinese

▢ Portuguese

‍

Good example

What is your favorite type of cuisine?

▢ Italian

▢ Japanese

▢ Chinese

▢ Portuguese

▢ Other

Tip 6. Avoid using negative questions in the multiple choice questions

‍

Bad example

You do not use which of the following

That might be read 'You use which of the following.'

If you can't avoid it, it’s better to stress the negative word like this: “You do not use which of the following.”

‍

Good example

You use which of the following

‍

Tip 7. Avoid biased questions

‍

Bad example

Do you think the politicians' salary is excessive?

▢ A little bit of excessive

▢ Moderately excessive

▢ Overly excessive

There are no wrong or biased questions. Always keep in mind that the questions should be as impartial as possible.

‍

Good example

Regarding the salary of politicians, it is

▢ very underpaid

▢ underpaid

▢ it's fair

▢ well paid

▢ excessively well paid

‍

‍

Tip 8. Vary the order and format of the questions

‍

Example A: How do you rate the quality of articles?

▢ very bad

▢ bad

▢ medium

▢ good

▢ very good

‍

Example B: How do you classify customer service?

▢ very good

▢ good

▢ medium

▢ bad 

▢ very bad

When possible, vary the order of the answers from a list. That will help minimize the participants' behavior for always choosing the list's first answer without reading the remaining ones. And vary the format of the questions: a checklist, following an open question or a Likert scale.

Take note that: do not vary the order when there is a list with a natural order like age or salary.

‍

Tip 9. Oh the jargon! Sweet jargon! Just avoid using it.

‍

Bad examples

How do you classify our app?

How frequently do you use our CRM?

‍

Good examples

How do you classify our mobile application?

How frequently do you use our customer relationship management software?

It would be best if you were attentive to the jargon used. You should know the target audience you're to who we're writing.

‍

‍

Tip 10. Think about  the sequence of the questions

Start with the easy questions first, and let the sensible and boring questions to the end, like the demographic ones.

‍

Test your survey!

With the survey created and the tips well applied, it’s time to test it!

Get feedback from your team members or other colleagues. They are a free sample, and they are pretty in hand! Find out if they have difficulties answering the survey or if they have any suggestions to point out! Take those insights into account, and review the survey if necessary.

I hope these tips helped you to write greater user research surveys. And don't forget: practice makes perfect!

References:

Designing for the Digital Age -  Kim Goodwin

User Experience Re-Mastered - Morgan Kauffman

Designing and Using Organizational Surveys - Allan H Church and Janine Waclawski