Quantitative Research

A scientific approach to UX design and research may seem intimidating. The word 'science' evokes an image of bearded geniuses tinkering in a lab, where dangerous objects await the unwary. But quantitative work is not just for scientists, as numbers are not only for mathematicians. Numbers and statistics can help improve digital products, user experience, and even user's life. When aimed at improving users' lives, it can be almost as creative and social as qualitative work.

This article is like a beginner's guide on why it's essential to use quantitative research in product improvement, user experience design, and how you can do it.

What is quantitative research in UX design?

Quantitative research quantifies user behavior and preferences to get insight into how users react to the product. This research method is the opposite of qualitative research, which focuses on understanding user behavior and emotions. Instead, the quantitative analysis aims at describing user behavior and preferences with numbers. It uses various tools, including surveys and online polls, to collect data from large groups of people. This type of research helps learn about the demographic information of users and what they want from a product/service.

In UX, we use quantitative research to gain insights into our user's expectations and understand their behavior as they interact with our product. We ask questions that can be answered with numbers (quantitative data) and analyze the results.

For example, if you want to know how many people are using your product daily, you could create an analytics dashboard that tracks usage metrics for each day. Or, if you want to see how long it takes users to accomplish a specific task using your product, you could run a usability test where participants are timed as they complete their assignment.

What is the difference between quantitative and qualitative research?

Quantitative research can tell you what people do, but it rarely tells you why. Qualitative research is usually a better bet if you want to understand why people do something. But if you're trying to know how many people are doing a particular thing, quantitative research is the way to go.

For instance, if you want to know whether or not your website or app is broken - and that's it - quantitative UX research is the way to go.

You might ask yourself:

• How many people are abandoning their carts?
• How many people are clicking on this button?
• Are more people buying after they use this feature?

But, suppose you want to know what motivates people to use your product, how they feel about it, and what is important to them. In that case, it's better to conduct qualitative research that will help you to get in-depth insights from users and potential customers.

To visualize the differences between quantitative and qualitative research, let's use the analogy of people shopping in the supermarket. We could imagine a technology that would count how many people enter the store, who they are (adults, children, elderly, women, men). We can analyze their cars left in the supermarket parking lot that show via car license plate from where they came. We could count how many items they bought, what they had in the shopping cart, and how much money they spent in the store on average. That's quantitative research. It shows us numbers and statistics.

However, when we would like to know, for instance:

• Why do they buy more broccoli than cauliflower?
• What motivates them to go to the store?
• How do they prepare themselves before shopping?
• What is crucial for them in choosing a particular product?
• How do they feel when they see the final amount to pay?

We should analyze this knowledge with qualitative research that would deepen the collected insights from numbers and statistics. Both types of research are essential to see the bigger picture of our user behavior.

To sum up, qualitative and quantitative UX research complement each other to provide the most valuable results.

What is a common goal of qualitative and quantitative research? It is a better understanding of users and creating a product that follows their needs.

When can you use quantitative research?

Quantitative research is appropriate in many situations where the goal is to learn about user behavior and attitudes. We can use it to make predictions based on large amounts of data, which can help product teams make informed decisions. We can apply this research early in the product development process to understand user needs or later in the process to measure satisfaction with features and plan its redesign eventually.

In UX research, quantitative techniques tend to be most helpful in exploratory research processes or at the end of a study to confirm or reject hypotheses collected through qualitative analysis.

We can use quantitative methods throughout the design process.

Still, it's important not to over-rely on them at the expense of qualitative research, which can provide deeper insight into underlying motivations and behaviors that may not be immediately apparent from numerical data.

As a result, quantitative research can help you find out, for instance:

• How many people use a product feature?
• What features do users like most about your product?
• How usable the page or the app elements are?
• How many clicks does it take for users to complete a task?
• How many people are getting stuck on specific pages?
• How long does it take for users to complete an action?
• What percentage of users will abandon your product after one session?
• What’s working, and what should be improved?

What are some of the most common quantitative UX research methods?

Quantitative UX research methods collect numerical data about users and help us measure, understand, and prioritize user behavior. Most of those methods are designed to yield data you can use to identify patterns and trends among a target population.

Here are some of the most common quantitative UX research methods:

1. Quantitative Usability Testing

2. Surveys and Questionnaires

3. Web and App Analytics

4. A/B testing or Multivariate testing

5. Card Sorting

6. Tree testing

7. Clustering Qualitative Data

8. Desirability Studies

9. Eye Tracking Testing

10. Click Tracking

11. Mouse Heatmaps

12. Cohort Analysis

13. Funnel Analysis

How to choose which method fits the best to your needs?

When it comes to quantitative research methods, there are many options. Sometimes it can be challenging to choose the appropriate one. It’s impossible to tell you exactly which way is best for your company, but we will show you what questions you should consider before deciding. Those will help you determine how you would like valuable data for your product.

Those questions are, for instance:

- What do you want to know from the research?

If you would like to conduct general research that gives you data about the usability of a product, competitors, and what are the biggest problems of your product:

Choose quant usability testing, web analytics, or surveys.

If you are more interested in specific methods relating to, for example, how to structure information architecture, what users think about the actual design and its different versions, what grabs the most attention:

Choose A/B testing, card sorting, tree testing, desirability studies, mouse heatmaps, click tracking, or eye tracking.

- How much does the research method cost?

The research cost is one of the most critical factors for the company. The number of participants and the amount of time spent by researchers will affect the cost of your research. In addition, the tools used and the size of the research team can have a significant impact on prices. Costs across companies vary depending on their circumstances, so below are just estimations.

Lower-budget teams often turn to digital methods to recruit and test users, such as remote usability testing, online card-sorting platforms like OptimalSort, A/B testing, Multivariate Testing, and web or app analytics. So if you prefer cheaper methods, consider them.

If you are ready for a higher cost, you can consider in-person methodologies that require more of the researcher’s time: like in-person usability testing or in-person card sorts. The most expensive research is eye tracking and can be enrolled only by teams with big budgets.

Ask other questions as well:

• How much time can you spend on testing?
• How many resources do you have to conduct research? Is it difficult for you to collect data?
• Is it challenging to analyze the data you can manage?
• Do you prefer to research online or in-person?
• What context of use do you choose - live, not using the product, task-based, or any?

In the end, consider where you want to focus more in your case - on quantitative research methods or qualitative research methods.

Pros of quantitative research

The primary benefit of quantitative research is that it can identify trends among many participants. It can also help uncover insights that may be difficult for participants to articulate directly.

Some benefits of quantitative research are:

• It allows for generalization across the population and scales well with larger groups of people.
• It provides statistically significant data, which helps prove the effectiveness of specific design changes.
• It comes up with large amounts of data very quickly and at a low cost, allowing for the creation of valuable insights.
• It helps gather factual information about users' preferences, behaviors, and attitudes.

Cons of quantitative research

Quantitative research has some limitations:

• It does not explain these trends, nor does it tell how or why behaviors occur.
• It can only provide general answers (not specific ones).
• There may be issues with sampling bias (the sample size might not reflect the population).
• All questions are dominated by closed-ended ones, limiting what you can ask regarding creative or alternative reflections.

Conclusion

In this article, we reviewed the methods used in quantitative studies to understand their role in the research process. Given the importance of an unbiased approach to data collection and analysis, we have provided an overview of how quantitative researchers achieve that objectivity. We also discussed using surveys and experiments as data collection tools, detailing their strengths and weaknesses.

See the power of numbers in your quantitative research and use it to understand user behavior. It can help you create the best possible user experience and a successful digital product.