Guide on Unmoderated Usability Tests, Prototype Tests and Concept Tests
Usability testing is a powerful method to evaluate your product with your users. In this guide, we share everything you need to know to run successful usability tests that can provide insights that can help your team improve your product’s user experience.
As product people, we have all had moments such as the following: For weeks, your team has hustled to dedicate numerous hours brainstorming and creating your product, ensuring it fits all the specs and all the core features are nailed down. However, in the back of your mind you keep thinking: "Will our users understand how to use this?”
Usability testing is the right solution to use when situations such as this one arise, as it can effectively help reveal core issues with your product’s usability, intuitiveness and value to your target users. Even when things seem very intuitive to the team, your users may have a completely different reaction since they weren’t building out the product from scratch. Validating your team’s assumptions and maximizing usability testing throughout the product development lifecycle can help catch issues quickly and help you continuously manage your product’s user experience.
Elevate your continuous UX Research
Validate your concepts and collect post-launch feedback through Hubble
What is usability testing and how is it useful?
Usability testing is a research methodology that can help measure your product’s user experience. Product and design can use usability tests to optimize the ease of use as well as the intuitiveness of a product.
Through a usability test, product teams can validate their hypotheses and test their assumptions with real users even in the earliest stages and the usability test results can be used as data points to identify any issues that can be addressed to ensure that users can get the most value out of your product and use it without any usability related challenges.
Through well-designed usability tests, successful product teams can build amazing products with delightful user experiences that not only solve their user’s problems but help them do it more easily.
Usability testing is just one of many different types of user research, aimed to evaluate the usability of a product. There are several other types of user research that are different than usability testing
- User Interviews: Conduct one-on-one or group interviews with users to explore their experiences, opinions, and pain points.
- Focus Groups: Bring together a small group of users to discuss specific topics or product concepts, facilitating group discussions and idea generation.
- Surveys: Collect quantitative data from a large number of users to gather insights on their preferences, demographics, and attitudes.
Types of Usability Testing
Usability testing comes in various shapes. Aligning with your team and understanding what your goals are will help you decide on the best method that can deliver the best insights. Broadly speaking, these are the different categories of usability tests that can be helpful for your projects:
- Moderated vs Unmoderated
- Remote vs. In-Person
- Quantitative vs. Qualitative
Moderated vs. Unmoderated
Moderated and unmoderated usability tests are two common approaches to conducting usability testing, each with its own advantages and considerations.
In moderated usability tests, a facilitator (usually a researcher or usability expert) is present during the testing session. The facilitator guides participants, observes their actions, and asks questions in real-time.
In unmoderated usability tests, participants complete tasks independently without a facilitator present. They follow predefined instructions and tasks provided by the researcher using unmoderated usability testing tools to record their actions and behaviors
Remote. vs In-Person
User research can be conducted either remotely or in-person.
As the word suggests, remote usability testing can be moderated or unmoderated. Remote usability testing is a method where participants evaluate a digital product or interface from their own locations, typically using their own devices and internet connections. This method has gained popularity due to its flexibility, cost-effectiveness, and ability to reach a geographically diverse user base.
On the other hand, In-person usability testing is a usability testing method where participants evaluate a product or interface in a controlled environment, typically a usability lab or a user testing facility. In-person testing involves face-to-face interactions between participants and researchers, allowing for real-time observation, facilitation, and feedback collection. Although remote testing can be more scalable and cost-friendly, in-person testing
What are the benefits of usability testing?
If you have not conducted any usability testing before, you may be asking, why is usability testing important and what benefits does it provide? Usability testing is a crucial component of the user-centered design process and offers numerous benefits for product development and improvement. Here are some of the key advantages of usability testing:
- Identifies Usability Issues
- Improves User Satisfaction
- Optimizes Design and Functionality
- Saves Time and Resources
- Supports Data-Driven Decisions
- Enhances User Engagement
1. Identifies Usability Issues
“Pay attention to what users do, not what they say”
Jacob Nielsen
First and foremost, usability testing helps uncover specific usability problems and challenges that users encounter when interacting with your product. This includes issues related to navigation, layout, content, functionality, and more - you can actually observe what your users do to uncover valuable insights that can help you improve your product.
2. Improves User Satisfaction
By addressing the usability issues identified during testing, you can enhance the overall user experience. A more user-friendly product leads to increased user satisfaction, trust, and loyalty.
Even if we think we understand our users, talking to users and testing your assumptions helps you continuously understand their needs and improve the product experience.
3. Optimizes Design and Functionality
Usability testing is a valuable tool for optimizing the design and functionality of a product or interface. By systematically gathering user feedback and observing their interactions, usability testing helps identify areas for improvement and informs evidence-based design decisions.
Design optimization is a continuous effort, and usability testing can help your team identify key issues that can be improved to enhance your product quality.
"Teams need to make a commitment to continuous improvement, and that means not simply refactoring code and addressing technical debt but also reworking and improving user interfaces. Teams must embrace the concept of UX debt and make a commitment to continuous improvement of the user experience.”
Jeff Gothelf
4. Saves Time and Resources
When you run usability tests, you can save time and money by avoiding UX mistakes. For example if you are an e-commerce company, and you find through usability testing results that your users are struggling with checking out, on your new checkout flow, it's significantly cheaper to make changes early than after when the product has already been shipped.
5. Supports Data-Driven Decisions
Usability testing provides both quantitative and qualitative data about user behavior and preferences. Audio and video recordings can help unlock qualitative feedback while heatmaps, time-on-task and completion rates can provide quantitative insights.
The insights that can be collected through usability testing can support evidence-based design decisions and guide feature prioritization.
6. Enhances User Engagement
Usability testing helps you to identify potential usability issues and make improvements before releasing a new product or feature. This can be hugely helpful in building better user experiences, more consistently, leading to increased user engagement.
Elevate your continuous UX Research
Validate your concepts and collect post-launch feedback through Hubble
When should I conduct a Usability Test?
Usability testing can be conducted across the product development cycle. Continuous testing allows the product team to make improvements to the product and never stop focusing on the end user and their problems. If you need step by step instructions on how to start building a Hubble usability test, please check out this guide from our help center and this blog post.
Here are the moments when you should consider running a usability test
- Early Concept and Ideation Phase
- Mid-Design and Prototyping Phase
- Pre-Launch or Beta Testing
- Post-Launch and Continuous Improvement
1. Early Concept and Ideation Phase
Through early usability tests and surveys, you can early on pinpoint user requirements, anticipations, and possible areas of frustration. Using this information as a baseline, you can initiate the design process with a strong basis for crafting a user-focused product that is even more user-friendly.
“Testing with one user early in the project is better than testing with 50 near the end.”
Steve Krug
2. Mid-Design and Prototyping Phase
Usability testing can be valuable when you have created higher-fidelity prototypes or interactive mockups. Testing at this stage allows you to identify and address usability issues before committing to full development.
3. Pre-Launch or Beta Testing
At this stage, you already have a near-final or beta version of your product. In this time, you can use usability testing to evaluate its overall effectiveness. This ensures that you can fix mistakes and make critical adjustments before the product is launched into the hands of users.
4. Post-Launch and Continuous Improvement
Even after launching and having a live product, it’s important to continue to run usability tests to prioritize your product roadmap. As the product continues to mature, you can run usability tests with your own users and with external testers to find potential issues caused by new feature updates and changes in user behavior.
How to create and conduct a usability test
Any team, can conduct a powerful usability test using a tool like Hubble.
- Define a goal and target audience
- Choose a user research tool
- Establish evaluation criteria
- Write a usability testing script
- Be mindful of test length
- Run a pilot test
- Recruit test participants
- Run the test with best practices
- Analyze and report your findings
- Iterate and repeat
1. Define a goal and target audience
Clearly outline the goals and objectives of the usability test. What specific aspects of the product's design or functionality are you looking to evaluate or improve?
As a next step, determine the target user demographic for your product.
A good set of questions to answer would be the following:
- What’s your targeted demographic (e.g. age, location, profession)?
- What do you want your testers to accomplish with your product? (i.e. purchase an item, book a home, add a new integration)
- How long should the task take from start to finish?
With answers to the questions above, draft a problem statement that you are trying to find an answer to through the usability test. For example: "Assess the usability of our new web portal for 30 - 40 year olds to book a vacation rental home." Having a clear problem statement will help you
This will help you determine the right research methods and metrics for evaluating the users’ experience.
2. Choose a user research tool
Choose a tool that your product team can use to run usability tests across the product development cycle. Hubble allows for your team to continuously conduct usability tests in all important phases including the early ideation, prototyping, beta testing and post launch.
Prototype Testing: Integrate with Figma prototypes and build usability tests to validate your designs. Check out this help center article to learn how to optimize prototypes for testing purposes or take a look a this example from Figma.
Participant recruiting: Recruit from more than 4 million participants in 150 countries
Card sorting: Create tests to understand how users organize and categorize information or content in a way that makes sense to them.
In-product surveying: Launch surveys in your product experience to collect live feedback from your product experiences
3. Establish your evaluation criteria
Before launching the usability test, determine what you would like to measure through the usability test. There are several metrics that can be collected through a tool like Hubble including:
Depending on the goal of your test, consider evaluating key usability metrics like:
- Direct and indirect success
- User satisfaction score
- Task completion rate
- Time spent on task
- Misclicks
Going back to the vacation home rental example, you can measure the usability of a checkout process, by reviewing metrics like the time it takes for users to complete the booking and overall success rate (i.e. percentage of successful rental bookings). It’s also a good idea to establish benchmark test results using earlier prototypes or competitor products, so you can track user performance as your product evolves over time.
4. Write a usability testing script
Every good usability test starts with a solid script that outlines the tasks to be performed by the tester. The script serves as the main guideline for the test participant and provides an opportunity to identify any potential problems or areas of improvement. Check out this guide to learn how to write good usability testing scripts.
5. Keep the tests concise
Please make sure that your tests are concise. The duration of a usability test should be influenced by factors such as the complexity of the product, the scope of the testing goals, and the level of participant engagement. However, as a general rule, usability tests typically last around 15–20 minutes and involve 5–10 tasks. This timeframe strikes a balance, keeping participants engaged while allowing your team to collect valuable insights.
6. Run pilot tests
To ensure that your test is ready to launch, get some of your team members to participate in the test. To remove cognitive bias, try to involve team members that are not directly involved with your project because ideally they should approach the usability test without having an idea of what it’s trying to validate. The pilot test will reveal any potential issues related with your script, the logic flows, the prototype loading time and other factors that may influence the outcome of your test.
7. Recruit Usability Test Participants
After building the usability test, you must recruit participants that pertain to the audience you are looking to target. Here is how you can find participants for your usability test:
Find testers from your user base
For starters, you can have your own product users as participants. There are multiple different ways to find the user participants, including in-product recruitment surveys, emails, and even phone calls. Testing with your own users does introduce some pros and cons; on the positive side, it can help you collect real, tangible feedback from people that are using your product every day. On the other hand, many of your users will have biases because they are already familiar with the product have cognitive bias regards to how the workflow should look. Using Hubble’s in-product SDK, you can opt in users to start building an internal participant group.
Recruit from a panel
You can also find participants from external pools using Hubble’s external recruiting services. Make sure to set up good screener questions to ensure high quality participants in the demography that you are looking for. Hubble’s external participant pool includes 3 million participants from more than 150 countries. You can find both B2B and B2C participants. For more information on recruiting participants effectively, please refer to our article on finding participants.
8. Create a positive testing environment for your testers
Always avoid leading questions, and create a comfortable testing environment so that you don’t introduce any bias into the results. Make sure your team is aligned on what questions should be asked and how each tester should be introduced to the usability test. The more honest your testers can be, the more informative the insights will be.
9. Share the test results with your team
Results from good usability tests will reveal a lot of insight that can elevate your user experience. Share your study results with other members of the product team so that you can identify UX issues and suggest potential improvements to continue upgrading your product’s user experience.
Some metrics you should be tracking include:
- Summary study metrics like total testers, bounce rate, misclick rate, and average duration
- User paths and funnels for how participants interacted, and optimal path analysis
- Direct. vs Indirect Success based on how many testers were able to complete the series of tasks
- Qualitative and quantitative questions with a visual representation of responses
10. Wash, rinse and repeat
Great product and UX teams continue to leverage Usability testing throughout the product development lifecycle. Just like products never stop evolving, usability tests should also continue to be iterated, evolved and revised as the product continues to change. Successful product organizations are able to grow their users and continue to gage the usability of their product by reassessing their priorities and adapting them so that it best servers their needs.
Elevate your product’s user experience by using usability testing
Usability testing is an amazing tool for any product team that can help to elevate a product’s user experience. Through usability testing, teams can understand their users better and unearth hidden insights that can provide guides to help prioritize the right initiatives to improve the product for its users.
Following the guides and tip above, usability testing will allow your team to:
- Discover UX pain points and issues
- Prioritize the right features on your product roadmap
- Understand how users are currently using your product
- Obtain quantitative and qualitative user feedback on your product’s features
If you need any help in implementing usability tests, or need any help in evaluating various tools, please try Hubble to help conduct continuous research across the product development cycle. Using Hubble, world class product teams have been conducting various types of research including building prototype usability tests, recruiting participants and launching in-product surveys, throughout different stages of product ideation and development.
