Ask eVOC

When should I use Lab-Based Usability Testing vs. Web-Based Testing?

Lab-based Usability Testing is conducted in-person, with a moderator and 1 or 2 participants, and can be performed either in-house or in a research facility. Usability labs are equipped with audio, video and picture and picture recording capabilities and most facilities include a two way mirror to allow clients to discretely view the live interaction between the moderator and the participant(s).

Lab-based testing is best when used to uncover low-hanging-fruit problems with user-interface design and to identify clear solutions for resolving those problems. Labs are also helpful when you want to understand the consumer's emotions or physical interaction with the site, such as mouse movement, facial expressions, or eye movements when using an eye tracking monitor.Since usability labs are conducted with a smaller sample (usually 6-8 sessions per day), they are qualitative in nature so the results cannot be projected onto a broader population.

Web-based Usability Testing is conducted online, using a browser or proxy based survey technology, that captures the natural behavior of participants as they complete tasks and answer questions online. Participants are invited into a study through an online intercept, email invitation, or third party panel, and they can complete the evaluation in their natural environment, such as their home, office or university.

Web-based testing is quantitative in nature and enables larger samples (200 participants or more) to complete an evaluation in a short timeframe. It also enables more advanced analytics and statistical significance testing which provides more reliable recommendations and projections.

In addition, Web-based testing is completed in the users' natural setting, from wherever they normally access the Web, so it eliminates any moderator bias or peer influence found in a multi-participant lab or focus group.

---

How important is sample size?

According to Jakob Nielsen, you only need to test with 5 users to discover 85% of the problems on your site . This is true when the focus of your research is very tactical - focused on learnabilty, efficiency, error prevention - and if you're testing a fairly homogenous sample of users.

However, if your study is more strategic - focused on customer experience benchmarking, making population projections, or measuring the ROI from your online initiatives - it is important to test with a large sample of users to provide statistically reliable metrics for strategic decision making.

In addition, if you're interested in learning how distinct segments interact with the site (e.g., patients vs. physicians vs. caregivers) or if you're comparing the needs of different user groups based on demographic or psychographic information, it is necessary to have a much larger sample to enable segmentation.

Most qualitative (lab-based) research includes 6-8 sessions per day and can include 1-3 participants per session. Most quantitative (web-based) studies begin at a minimum of 200 participants per study and can range up to 5000+ depending on the breadth information collected and the type of survey method that is used.