How to create effective unmoderated concept tests

October 18, 2023

‍

Unmoderated concept testing is a very popular UX research methodology used by modern product teams that iterate and validate continuously. Unmoderated concept testing tools like Hubble can be used to validate feature ideas and collect tangible user feedback in the early stages of product development. Without spending significant engineering resources, product teams can test their assumptions and collect specific product feedback from their target audiences. However, building an effective unmoderated concept test requires effort and planning. In this post, we would like to share some of the key tips that we provide to our customers when they get started with their first unmoderated research.

‍

‍

Align your goals with other team members

Delivering a successful unmoderated concept test starts by ensuring that the team is aligned well on the goals. Discuss with your team and make sure that everyone understands what needs to validate through the unmoderated concept test. We believe this is particularly important when there are multiple functions involved in the process like product, design and research. Collaborating from the beginning will help teams drive the best research outcome and ensure that the results can drive the best action forward. If you need tips on how to collaborate best beteween UXR and product, please see this blog post.

Making sure that the entire team is aligned is a key ingredient for success in any UX research method.

‍

Develop a plan for unmoderated test recruitment

In all types of user research, recruiting participants is one of the most important elements to consider. Teams can either recruit unmoderated concept testers directly from their active user base or leverage a pool of external participants that can help them test with individuals that are not familiar with the product. Recruiting actual users can be advantageous because they will have a strong context around the product and will be able to test more nuanced product-specific questions in the unmoderated concept test. On the other hand, external panels can expedite the research process because collecting responses from them usually requires less turnaround time. Hubble customers have the option to run both, and our amazing customer success team will assist in both strategies 😎

‍

Create a dedicated prototype for testing purposes

Using a well crafted Figma prototype is important to create a good experience for unmoderated concept testers but also ensure that the results are effective and value additive. We highly recommend creating frame based prototypes, and creating a separate panel to build a test prototype because it makes the importing process much smoother and snappier. We also recommend adding user paths and have a clear ending screen so that you can properly track key quantitative statistics such as success rate as well as time spent on task. If you haven’t checked out, here is a guide on how to optimize your prototypes for concept tests.

‍

Well designed unmoderated studies can uncover high quality insights for your product development process

‍

‍

Write a good script for your concept test

Writing a good script is extremely important to ensure a great outcome for unmoderated concept tests. When writing a good unmoderated test script, you need to maintain a fine balance of providing enough information and background knowledge for the tester to understand the purpose of the test, but also not providing specific step by step instructions that can negatively impact the quality of the unmoderated concept tests. For usability driven unmoderated concept tests, is better to provide general instructions and observe how the testers go through the flows to identify any issues and potential UX loopholes. If you need more specific advise on how to create a good unmoderated concept test script, please take a look at this blog post

‍

Test the unmoderated concept tests internally prior to publishing

Make sure to test the unmoderated concept test within your team internally before releasing it to actual participants. This helps ensure that there are no technical issues or roadblocks that can potentially negatively impact the results of the test. Hubble offers public preview links that you can share with anyone inside and outside your team.

Unmoderated concept tests are very powerful UX research tools that can unlock value to your team by allowing rapid & fast experimentation through the product development process. If you follow some of the tips that we shared above, your product team will be able to collect powerful user research insights through unmoderated concept tests.

Do you have any other suggestions or comments on how to succeed with unmoderated concept tests? Please email me at brian@hubble.team or contact us directly in our community πŸ˜ƒ

‍

Frequently Asked Questions

How often should I run unmoderated tests?

The frequency of running unmoderated tests can vary based on factors such as the product development stage, project timelines, and the need for continuous feedback. A general guideline is to incorporate unmoderated tests regularly throughout the product development cycle. The cadence of unmoderated tests should align with the iterative nature of product development and the desire for continuous improvement.

What is the added benefit of unmoderated tests over moderated tests?

Unmoderated studies provide flexibility in terms of timing, allowing participants to engage with the study at their convenience. This flexibility can lead to a larger and more diverse participant pool in a quick turn over. Additionally, unmoderated studies are often more cost-effective, as they require fewer resources for live moderation.

Nevertheless, the choice between moderated and unmoderated studies depends on specific research goals, the complexity of the study, and the desired level of control over the testing environment.

How long should unmoderated studies take?

The duration of unmoderated studies can vary depending on the complexity of the research objectives and the tasks participants are required to perform. In general, it's advisable to keep unmoderated studies relatively short to maintain participant engagement and prevent fatigue. Studies ranging from 15 to 30 minutes are often considered optimal, allowing participants to provide valuable insights without feeling overwhelmed.

How do I screen participants for quality data?

Make sure to clearly define the target audience that fits the needs of your research. Having specific criteria and desired characteristics allow the recruited participants to better align with the research objective. Translate the criteria into screener questions so that the ideal participants could be filtered through and recruited for the study.

To learn more about writing an effective screener, refer to creating effective screener surveys for quality participants.

Read other articles
Brian is the CEO and Founder of Hubble. Brian started Hubble to build a unified tool that allows product and UX teams to continuously discover their user's needs. Brian leads the sales and marketing efforts at the Company and he also works closely with the product team to deliver the best user experience possible for Hubble customers. In his free time, Brian likes to explore New York City and spend time with his family.

Related posts

Hubble vs. Maze

Hubble vs. Maze

13 best UserTesting alternatives

13 best UserTesting alternatives

Challenges and best practices for recruiting B2B participants in user research

Challenges and best practices for recruiting B2B participants in user research