UX Research 101: Guide to Surveys

Surveys are important tools in UX research that help you understand user experiences and preferences through structured questions. Surveys therefore play a crucial role for UX researchers, being one of the most widely used forms of research that can be used at multiple stages of the product development cycle, ranging from the discovery phase to the post-production phase when you need to gather user feedback and satisfaction.

Needless to say, surveys should form a fundamental component of your research toolkit, as it offers valuable user insights at a relatively low cost and time compared to other methodologies like in-depth moderated interviews and usability testing. Nevertheless, maximizing the benefits of surveys requires thoughtful planning and consideration both when creating the survey as well as when you are analyzing the results data. This guide presents some of the best practices and insights to build impactful surveys and how to best use survey insights for your research goals.

Elevate your continuous UX Research

Validate your concepts and collect post-launch feedback through Hubble

Schedule a demo

What are UX research surveys?

Surveys are structured data collection instruments used in UX research to gather feedback, opinions, and preferences from a sample of users. Usually, survey participants are presented with a series of questions designed to collect specific responses related to their experiences with a product or service. There are various ways of distributing surveys including online forms, emails, in-app and product prompts, or paper-based questionnaires. One of the most powerful advantages of surveys is that they offer quantitative data that complements qualitative research methods such as interviews and usability testing.

Some of the common objectives for running UX research surveys are:

• Exploring the problem space: Because it's relatively cost effective, surveys are efficient tools that can be used to explore a problem space by gathering information, opinions, and feedback from a targeted audience.
• Uncovering the what factors: Surveys help uncover the "what"  and the "why" by collecting data and insights to a particular topic or problem space.
• Understanding user needs: Surveys can help UX researchers gain insights on user preferences, pain points, and expectations, aiding in the creation of user-centric designs.
• Gathering feedback: Surveys provide a platform for users to express their opinions and provide feedback on products or features.
• Validation: Surveys can be used to validate hypotheses, test assumptions, and gauge user reactions to proposed design changes.
• Quantitative analysis: By collecting numerical data, surveys enable you to conduct quantitative analysis and draw statistically significant results.

‍

What are the advantages of surveys?

Surveys are a indispensable tool in user research. Some of the pros of using surveys in user research include:

• Versatility: Surveys are very versatile, meaning that they can be used during various stages of product development, from initial discovery to gathering feedback on new features or designs.
• Cost-effectiveness: Surveys are very cost effective relative to other research methodologies such as moderated user interviews, focus groups or usability testing.
• Scalability: Surveys can be administered to a large number of participants, making it an effective option when gathering insights from a diverse audience and driving statistically significant results.
• Efficiency: Surveys can be run with high velocity, allowing you to collect a large amount of data in a short period of time.

What are the disadvantages of surveys?

• Limited depth: Often times, surveys don't provide as much depth as other qualitative research methods like interviews or observation, because they rely on a set of predefined questions within limited time.‍
• Response bias: With surveys, there is always an inherent risk of response bias, as research participants may not accurately represent the broader user population, leading to skewed results.
• Lack of context: Surveys may not be able to effectively gather contextual information that can be obtained through direct interaction with users.
• Incomplete data: Just like other quantitative data, survey results include incomplete responses, which require an additional step of data cleaning before analyzing the data.
• Potential for misinterpretation: Without proper validation or follow-up questions, participants' qualitative or textual responses may be misinterpreted.

Despite some of the disadvantages listed above, surveys are great research methodology utilized in various stages of product development cycle.

How are surveys different from polls and questionnaires?

Surveys, polls, and questionnaires are often used interchangeably, but they serve different purposes. Surveys are comprehensive instruments designed to collect data on specific research objectives. Polls are typically shorter and focus on gathering quick feedback on a single question or topic. Questionnaires can be a part of a survey but are also standalone tools used to collect data on a range of topics.

Here are some of the key differences between surveys, polls, and questionnaires:

1. Surveys: Surveys are comprehensive research tools used to collect data, typically involving a series of structured questions designed to gather detailed feedback on various topics. Surveys are often used in UX research to gather insights on user preferences, behaviors, and experiences.
2. Polls: Polls are typically shorter and more focused than surveys. They usually consist of one or multiple questions and are designed to quickly gather opinions or preferences from a large audience. Polls are commonly used to gauge public opinion on specific topics or to gather feedback on a particular issue.
3. Questionnaires: Questionnaires are similar to surveys but are typically shorter and less comprehensive. They often consist of a set of predefined questions that respondents answer in writing. Questionnaires are commonly used to gather specific information or opinions from a large group of people.

‍

Different Types of Surveys

Surveys can be categorized into different categories depending on their structure and type of data they are designed to collect. Each type of survey has its own strengths and limitations, so depending on the team's research objectives, the nature of the questions as well as the approach of building a survey will change.

Quantitative surveys

Quantitative surveys use closed-ended questions and predefined response options, represented through radio button type questions, to gather numerical data that can be analyzed statistically. These surveys aim to quantify attitudes, behaviors, or opinions within a population, typically involving large sample sizes, structured questionnaires, and statistical analysis.

Quantitative surveys allow for generalization of findings to a larger population, precise measurement of variables, and the ability to identify patterns or correlations in data. Some of the most common use cases for quantitative surveys inlude: 1. measuring customer satisfaction and assess product usability.

Qualitative surveys

On the other hand, qualitative surveys use open-ended questions and free-form responses to gather rich, descriptive data about participants' experiences, perceptions, and attitudes. Unlike quantitative surveys, qualitative ones involve smaller sample sizes, and qualitative data analysis techniques such as thematic analysis.

Qualitative surveys and quantiative surveys are both important for product teams. Qualitative surveys offer deep insights into users' motivations, preferences, and behaviors, allowing researchers to explore and uncover user motivations and behavioral insights, used to understand users' needs, pain points that bring ideas to improve the user experience and build additional features into the product.

Mixed-methods surveys

Mixed-method surveys combine elements of both quantitative and qualitative, allowing researchers to combine the benefits of both into one. Mixed-method surveys usually involve the sequential or concurrent collection of both quantitative and qualitative data, followed by integration or triangulation of findings during analysis.

These surveys offer a comprehensive understanding of research questions by triangulating data sources, validating findings, and providing a more holistic view of user experiences. Because of these benefits, mixed-method surveys are often used to corroborate quantitative findings with qualitative insights, provide context to numerical data, or explore unexpected findings in more depth.

Cross-sectional surveys

Cross-sectional surveys collect data from a sample of participants at a single point in time. This form of surveys lets you examine relationships between variables and make inferences about the population at that specific moment. Cross-sectional surveys often involve administering surveys to participants from different demographic groups or segments to capture a broad range of perspectives. Data collected are analyzed to identify patterns, trends, or associations.

Longitudinal surveys

Longitudinal surveys collect data from the same participants over an extended period, allowing you to track changes, trends, or developments over time. This type of surveys enables the study of individual trajectories and the identification of causal relationships. Participants are surveyed multiple times at regular intervals, ranging from weeks to years, to assess changes or stability in variables of interest.

In-product surveys

In-product surveys are integrated directly into a digital product or website, typically presented to users while they are actively using the product. These surveys aim to collect contextual feedback, opinions, or user experiences related to specific features, functionalities, or interactions within the product.

In-product surveys are often triggered based on user behavior, such as completing a specific task, visiting a specific URL page, or spending a certain amount of time on the platform. In-product surveys can be presented through various formats, including pop-up dialogs, slide-out panels, or embedded forms within the user interface. In-product surveys are commonly used by product teams to gather feedback on new features, user satisfaction, usability issues, or product-market fit. They can also be used for onboarding or user training purposes to assess user understanding and effectiveness of instructional materials.

Transactional surveys

Transactional surveys are sent to users immediately after completing a specific transaction or interaction, such as making a purchase, completing a support ticket, or signing up for a service. These surveys aim to collect feedback on the user's experience particular to the transactional process.

Transactional surveys are typically triggered automatically based on predefined events or actions, such as order completion, service cancellation, or customer service interaction. They are often delivered via email or in-app notifications and contain questions related to the specific transactional experience. Transactional surveys are widely used in e-commerce, customer support, and service industries to measure customer satisfaction, gather feedback on service quality, or identify opportunities for upselling or cross-selling. In addition, transactional surveys are also used in subscription-based models to assess user retention and churn factors.

There is no one-size-fits-all solution for surveys. Each type of survey has its own strengths and weaknesses, and the effectiveness of a survey type will depend on the research objectives, the target audience as well as the specific context of the study.

When should I run surveys?

Surveys should be strategically timed and aligned with the goals of your user research and product development efforts. Below are examples of when you should consider running surveys:

Product discovery phase

During the initial product discovery phase, surveys can work as valuable tools to explore initial problem spaces and understanding user needs and preferences before starting the actual product development process. Running surveys in the discovery phase helps gather early signals to inform product ideation and feature prioritization.

Iterative development

Surveys can be conducted throughout the product development lifecycle to gather feedback at various stages of product iteration. For example, you can run surveys after launching a new feature or product update to gauge user satisfaction, identify usability issues, or gather suggestions for improvement.

Onboarding experience

Surveys can be used during the onboarding process to assess user understanding of the product and its features. By collecting feedback from newly onborarded users, you can identify areas where additional guidance or support may be needed to improve the onboarding experience.

Data triangulation

Because surveys are easier to implement than other research methodologies, teams can consider using surveys to complement other research methodologies used in every stage of product development. For example, surveys can complement usability testing results by gathering quantitative data. Running surveys alongside usability tests provides a comprehensive understanding of user experiences and helps identify potential areas for optimization.

Post-interaction feedback

Surveys can be triggered after specific user interactions or transactions within the product, such as completing a purchase, submitting a support ticket, or attending a webinar. Collecting feedback immediately after these interactions provides contextual feedback on the user experience and allows for timely improvements.

Regular user feedback

Establishing a fixed cadence (i.e. quarterly or bi-annually) for running surveys on a regular basis, can help maintain ongoing communication with users and track changes in user preferences and behaviors over time. Regular surveys can also demonstrate a commitment to listening to user feedback and continuously improving the product.

How do I launch a Survey?

1. Define your survey objectives

First, start by clearly defining the objectives of your survey. What specific insights are you looking to gather? What research questions do you want to answer? Establishing clear objectives for your surveys help you design and launch your survey with higher precision and quality.

2. Identify your target audience

Determine the target audience for your survey based on your research objectives. Who are the key stakeholders or user segments you want to gather feedback from? Consider demographic factors such as age, gender, location, and occupation that may influence the responses.

3. Choose the right survey tool

Select a survey tool or platform that aligns with your research needs and objectives. There are many options available, ranging from simple free survey builders like Google Forms to more robust platforms like SurveyMonkey or Qualtrics. Choose a tool that offers the features and functionality you require, such as customizable question types, respondent targeting options, and data analysis capabilities.

At Hubble, we use in-product surveys to help product teams collect contextual, realtime feedback by prompting surveys to actual users that are engaging with your product and website. in addition, you can create usability testing surveys or collect quantitative NPS and CSAT scores.

4. Design your survey

Create a clear and concise list of survey questions that effectively captures the information you need to collect from your respondents. Start by drafting a list of survey questions based on your research objectives, keeping them focused and relevant to the topic at hand. Use a mix of question types, including multiple-choice, rating scales, open-ended, and demographic questions, to gather both quantitative and qualitative data.

Below are some important checklists for designing your survey:

• Ensure that every question in the survey is intentional and purposeful. If there are too many questions, reconsider why each one needs to be asked and what insights you hope to gain.
• Keep the survey concise and relevant to avoid survey fatigue.
• If a question contains too much information or complexity, consider breaking it down into smaller, more digestible pieces.
• Avoid binary yes-no questions; instead, provide more specific options to learn about participants’ behaviors.
• Ensure that the type of question (single select, multi-select, numerical scale, etc.) is the most appropriate option for each question.
• Check that the wording and nuance of the questions are unbiased.
• Avoid biasing the questions by giving away important keywords.
• Use skip logic as needed to facilitate the survey's flow.

5. Pilot test your survey

Before launching your survey to your target audience, conduct a pilot test internally or with a small group of participants to identify any potential issues or ambiguities in the questionnaire. Review their feedback and make necessary revisions to improve the clarity and effectiveness of the survey. Survey platforms like Hubble contain ways to preview your survey.

6. Set up distribution channels

Determine the most appropriate channels for distributing your survey to reach your target audience. This could include email invitations, social media posts, website pop-ups, or in-app notifications. Consider the preferences and behaviors of your audience when selecting distribution channels to maximize response rates. Use external survey participant pools like Hubble's participant pool to be able to find specific personas if you are looking for external participants.

In order to encourage participation in your survey, clearly communicate its purpose and value to potential respondents. Highlight any incentives or rewards for participation, such as discounts, prizes, or access to exclusive content. Use persuasive messaging to motivate respondents to complete the survey and emphasize the importance of their feedback.

7. Launch your survey

Once everything is set up, launch your survey to your target audience. Monitor the distribution process closely to ensure that the survey reaches the intended recipients and that any technical issues are addressed promptly.

The time it takes to achieve the desired sample size depends on the participant profile. Keep track of response rates and monitor the progress of your survey when you can. If response rates are lower than expected, consider sending reminders or follow-up communications to encourage participation.

8. Analyze and interpret results

As data become available, analyze the collected data to extract meaningful insights and actionable findings. Use statistical analysis tools or qualitative coding techniques to identify trends, patterns, and themes in the responses. Interpret the results in the context of your research objectives and use them to inform decision-making and drive improvements in your product or service.

‍

Important Steps to Analyzing Survey Data

Analyzing survey data involves several key steps to derive meaningful insights.

Revisit research objectives

Revisit the original research plan and objectives, and take a skim through the data that has been collected. Clearly outline what you aim to achieve with your analysis. Determine the key questions you want to answer and the insights you hope to uncover.

Data preparation and cleaning

Before diving into analysis, ensure your data is clean and organized. Remove any incomplete or irrelevant responses. Depending on the distribution channel, the quality of the data may vary and require more manual inspection of cleaning data, which mainly includes removing incomplete responses, redundant submissions, and formatting textual data. For quantitative data, use spreadsheets or other data analysis tools like Airtable.

Explore relationship and identify patterns

Explore relationships between different survey variables. Use cross-tabulations to examine how responses vary across demographic groups or other relevant segments. For more details on quantitative survey analysis, see below. Additionally, look for patterns and trends in the data. Use visualizations such as charts, and graphs to identify patterns and trends visually.

‍

Quantitative Survey Analysis

Quantitative data analysis techniques can provide valuable insights when working with survey data. Depending on the type of data that you have, analysis approach will vary. Here are some common methods from basics to more advanced:

Descriptive statistics

Begin by conducting descriptive analysis to summarize the main characteristics of your data. Calculate basic descriptive statistics such as frequencies, percentages, means, medians, and standard deviations to summarize the distribution and central tendency of survey responses.

Correlation analysis

Conduct correlation analysis to identify relationships between continuous variables. Determine whether changes in one variable are associated with changes in another variable and the strength of these relationships. Correlation analysis is fairly easier to do compared to other more advanced methods.

T-test

Perform t-tests to compare means between two groups and determine whether there are statistically significant differences. This method is useful for analyzing survey data with a binary or categorical independent variable.

Chi-square test

Conduct chi-square tests to examine the association between categorical variables. This technique is commonly used to assess whether observed frequencies differ significantly from expected frequencies.

Regression analysis

Use regression analysis to explore the relationship between one or more independent variables and a dependent variable. This technique can help identify predictors of specific outcomes and quantify their effects.

Analysis of Variance (ANOVA)

Use ANOVA to compare means across two or more groups and determine whether there are statistically significant differences. This method is particularly useful for analyzing survey data with categorical independent variables.

Time-Series Analysis

If your survey data includes repeated measurements over time, use time-series analysis techniques to explore trends, patterns, and changes in responses over time.

‍

Qualitative Survey Analysis

Analyzing qualitative data in surveys involves a more systematic approach to examining and interpreting text-based responses to open-ended questions or comments. Unlike quantitative data, which can be easily summarized using statistical methods, qualitative data requires a more in-depth and subjective approach to analysis. To learn more in depth about analyzing qualitative data in user research, refer to our comprehensive guide on qualitative data analysis.

Here are some key steps to effectively analyze qualitative data in surveys:

Familiarizing the data

Begin by familiarizing yourself with the qualitative data collected in the survey. Read through all the responses to gain an understanding of the range of perspectives, themes, and patterns present in the data.

Coding

Organize the qualitative data by coding, which involves systematically labeling or categorizing responses based on their content or themes. Coding helps to identify recurring topics, ideas, or sentiments within the data set.

To learn more about coding, refer to our guide on coding in qualitative data analysis.

Thematic analysis

Conduct thematic analysis to identify overarching themes or patterns that emerge from the coded data. Look for commonalities, differences, and relationships between codes to develop a deeper understanding of the respondents' experiences, opinions, and behaviors.

To learn more about thematic analysis, refer to our guide on thematic analysis and affinity mapping.

Data triangulation

Once you have identified the themes, compare the qualitative data results with quantitative data. Validate the qualitative findings by comparing them with other data sources or perspectives, such as quantitative survey data or external sources of information.

Interpret the qualitative findings by synthesizing the themes, patterns, and insights uncovered during the analysis. Explore the implications of the findings for the research objectives, and consider how they can inform decision-making, problem-solving, or further research efforts.

In this guide, we explored how to use surveys for UX research. Surveys are important instruments that cannot be excluded from any UX research tool set. Because of their versatility and relatively low cost of execution, they can be used widely across the product development process to gather insights from users and maximize quantitative and qualitative data points on user behavior and satisfaction. In the following guides, we will dive deeper into some of the specific types of surveys that we mentioned above, and how you can use them effectively for your projects.

‍