In the past year we have seen an increasing demand for multi-device usability testing. Multi-device usability testing involves asking users to perform tasks on more than one device in a single session; devices may include desktop computers, tablets, and mobile phones. Testing multiple devices in a single session has been useful when testing Responsive Web Designs, as well as testing the fluidity of a brand’s experience from desktop websites to mobile and tablet websites, and/or mobile apps.

One benefit to conducting testing across devices in one session vs. testing designs on a single device per session is that the user is able to shed light on the overall experience and consistency across devices. How fluid is the experience? How easy is it for the user to transition from desktop to mobile and vice versa? Not only will testing multiple devices with one user provide insights into an overall experience, but it can also give direction and insight into how and why users may gravitate to one device for certain tasks vs. another.

What we have learned from conducting these tests is that preparation and flexibility play key roles in making sure this type of testing goes smoothly. Below are a few tips to help you prepare for and conduct multi-device usability testing:

Recruiting

• Recruit to reflect your real audience: If 90% of those using your mobile app are iPhone users, then testing should focus on iPhones vs. recruiting an even mix of iPhone and Android users.
• Forget the Gadget Lab: We have seen greater success in this type of testing (and mobile-only testing) by having the user bring in their own device. Not only is the user more confident navigating the device, but often there are added insights. With the users’ own device, you may get to see what apps the user has downloaded and how they have organized the information on their phone. Just be sure to clearly specify the types of mobile devices, model and version of software / operating system (e.g., FROYO, GINGERBREAD, etc.) required for testing during the screening process.
• Get the OK to Download: If users will need to download anything on their phone or tablet, it is best to ask this during the screening process to avoid any trouble down the road.

Discussion Guide

• Counterbalance Starting Device: Typically we will start an equal number of participants on each device (e.g., N=12 / 6 start on desktop, 6 start on mobile), unless the team is focused on the usability of one more than the other. We also make sure that we have an even mix of participant types (if research has segments) for each starting device. For example, if we have a total of 12 participants, with 6 Prospects and 6 Current Users, then 3 Prospects and 3 Current Users will start on desktop and 3 Prospects and 3 Current Users will start on mobile.
• Repeating Tasks is OK: It is okay to repeat tasks across devices; we have found that usability issues are still uncovered. Users will have expectations based on their initial experience, but this happens in real life, too! It is nice to add some variety to the tasks if possible, but do not stress if prototypes only allow for the same tasks to be performed on the various devices. By shuffling the starting device, the team will still be able to get initial impressions on the experience from a portion of the users. In fact, if you are looking for an apples-to-apples comparison across devices, it is important that you have the user perform the exact same task on each device.

Set Up

• Wi-Fi Ready: We typically ask users to join the Wi-Fi network on their phones or tablet prior to starting the session to save time. If you are conducting labs at a facility, it’s helpful to ask the front desk to share the Wi-Fi information to the participant upon arrival.
• Bring a Back-Up Device: Yes, we did say to forget the gadget lab; however, it does not hurt to have a back-up device for unpredictable technical difficulties. Typically, we just bring in the most popular mobile / tablet device used to access the site/app in the case that the user’s device fails to work during the test.
• Device Hot Spot: To ensure that the respondents’ actions on their mobile device are captured, it is best to designate an area for the camera to target. Typically, we tape off the area where the users should keep their device in order for it to be successfully captured on camera.
• Capture All Angles: Many of our clients view sessions remotely and it is a much more engaging experience if all angles of testing are captured, which includes the respondent’s face at all times and then the desktop screen, mobile phone, or tablet. In order to do this, we typically have multiple cameras in the room, which can be controlled by our video technician in the backroom. Our video technician is on site during the sessions to switch between cameras during testing to make sure all angles are captured at the appropriate times.

Conducting the Usability Session

• Keep it Real: One thing to avoid in user testing is forcing the participant to complete tasks on a device he or she is not familiar with or would not use in real life.
• Ask Wrap-Up Questions: One of the biggest benefits to multi-device usability testing is the ability to understand the overall experience across devices, so don’t forget to ask questions about this! Some questions you might ask are, “How does the desktop experience compare to the mobile experience?” or “Are there any tasks you prefer to do on one device vs. another?”

That sums up our tips for preparing and conducting multi-device usability testing. As each research project is unique, there are always ways to refine and adjust the methodology to ensure that the research objectives for your project will be met. If you have any questions, please do not hesitate to reach out to us at .

We have done a number of projects lately with sites that need to look at how they greet customers when coming to their site and this got us thinking about landing pages. Very often, users are not entering your site through the home page any longer. Ads, emails and searches are often driving users to your site. With over 1 billion searches per day on Google alone, you should be giving serious thought to the way users are entering your site and whether or not you are putting your best foot forward when they do. If you don’t greet visitors in the best possible way, your goal - conversion - will not be met.

In this post, we’ll take a look at the different ways a user may be entering your site and best practices for these entry points.

The most obvious entry point is your home page. A user may be familiar with your site, may have seen your URL somewhere, or searched for your site and ended up on your home page. The best home pages:

• Make it clear that they are home pages - a user should never have to guess
• Provide access to main navigation, not only to allow for easy access to information, but also to provide an indication of what your site offers
• Indicate the site’s value proposition or benefits - identify yourself to new users
• Offer a clear path for returning visitors to access their account or personalized information
• Entice users into the site - with images, topics of interest, etc. to encourage further interaction with the site

Today, we want to focus more on the pages users reach when they enter your site via a campaign or search. This may be a page specifically designed to be a landing page or simply a page on your site that is a default landing page because of the amount of search traffic driven to it. There are a number of things that should be done whichever type of page users are landing on to greet them to your site. The tips for home pages above apply, as do the following:

• Ensure consistent branding
  • Your logo should be visible and look like your company’s logo - a user should never have to guess whether or not they are in the right place
  • The rest of the branding - look, feel and design - should be consistent with the feel of the rest of your website. This doesn’t mean a campaign or page cannot use different elements than the rest of your site, but the overall look and feel should not be so different that users are again left guessing where they are or are confused if they navigate to the rest of the site
• Provide a clear value proposition that is consistent with both how the user entered the page and the breadth of content, products or services your site offers
  • You want to be sure you are delivering on the specific promise you made in your campaign or search result; the page that users enter should match any expectations you have set for them
  • But, you also want to be sure you do not miss an opportunity for users to understand all of what you have to offer, as something else you offer may pique their interest
• Provide a clear call-to-action
  • This is true whether you are requesting interaction, such as registration from a user, or if you simply want them to explore your site further
  • You want to provide clear avenues to interaction and highlight them with clear instructions to users, so employ action-oriented terms: “Get Your Free Quote”, “Learn More”, “Contact Us”, etc.
• Keep your design clean
  • While you want to make sure you are providing options to users, you do not want to overwhelm them or distract them from your message
  • Include links to key pages or navigation (this page should not be a trap), but do not include every offer or marketing message you want a user to see on this one page
  • Focus on the expectations set and deliver on those on this page using bulleted, easy to scan benefits or value proposition information
• Be aware of the fold
  • Key content and call-to-action messaging needs to be immediately visible to users
  • If users have to scroll for a payoff, you have lost them, so consider who is coming to your site and on what devices / screen resolutions and design to make sure they do not have to scroll to see key information

Below, we will look at some of the best practices specific to different types of landing pages, as well as examples of what to do and what not to do when welcoming a visitor to your site.

CAMPAIGN-SPECIFIC LANDING PAGES

1. While I mentioned it above, I will mention it again here, because this is so important for campaign-specific landing pages: You must deliver on the expectations set by whatever it is that sent a user to the page.

• In the example below, a search for Best Buy, offers a link to what it says is Best Buy’s official site. However, the link actually takes visitors to a landing page specific to Best Buy’s deals and weekly ad. The effect is disorienting for a visitor looking to access the Best Buy site because the set-up does not meet expectations.

• On the other hand, this AT&T ad on CNN.com provides access to the content that was specifically detailed in the ad, while also providing access to a wider range of options if a user is interested in other products or services. This means the page delivers on both the specific user need and a wider range of interests they may have.

2. If you are requesting information from users – to fulfill an offer, get them started using your site, or provide them with customized information, your task with a landing page is even harder. You may lose people as soon as they see a form to be completed. To counteract this:

• Your page should load quickly so users see the whole picture very quickly
• That whole picture should include a clear statement of benefits: what is the value exchange you are offering? You are asking for their time and personal data - what are they getting from you in return?
• What type of time commitment are you asking of them? Is all the information you are seeking on one page or is this a multi-step process?
• What will the payoff be? You need to set specific and accurate expectations for what will happen at the end of the registration process or form.

3. Additionally, any forms provided on campaign-specific landing pages should adhere to form best practices (see more about best practices for forms here). Some highlights:

• Be specific about what you are asking for and WHY you are asking for it
• Ask for only the most important information, focusing on things that are least invasive, i.e., if you can use only a zip code for location, do not request exact street address. You can request that information once you have deepened the relationship with the customer
• Include logos or icons that highlight security of information. Logos like Verisign, Truste and the Better Business Bureau, as well as icons, such as a picture of a lock, reassure users that they are using a secure site and that the information provided will be safe

4. Examples in the auto quote space provide an interesting contrast in what works and what doesn’t:

• Progressive provides a focused page that directs users to a call to action, while addressing critical user information needs:

• Geico, on the other hand, overwhelms users with information. This takes focus away from the purpose of the page and does not address user information in a concise way, making it less likely for users to convert.

DEEP LINKED PAGES

1. For pages on your site that become landing pages because the content is often searched for and makes it a key entry page to your site, it is important to let users know where they are on your site

• Many times, breadcrumbing can be used to help orient a visitor (for some sites, like those with a lot of dynamic content, breadcrumbing may not be the best option. For more information on breadcrumbing best practices, see our blog on breadcrumbing)
• Additionally, clear highlighting within the navigation can be used to indicate position
• This example from CNET shows that a search for “best tablet computers” delivers users to the specific content requested, but also provides good insight into where a user is and what else might be available

2. Access to full site content, especially the home page, is even more important

• Even though the page delivered may match users’ expectations exactly, you want to be sure they can move up in your site’s hierarchy easily to get a wider view
• Ensure access not only to the full navigation, but also link to other information related to that specific page so users can move easily to other content they might find interesting
• Even with these additional links, remember to initiate a clear call-to-action on the page a user reaches
• In this example, searching for “hotel Rome” leads users to a specific hotel property, which offers a clear explanation of how they got to the page and how they could explore the hotel information or other destinations. It also has an immediate call to action above the fold:

Using what works and what doesn’t from these best practices and examples, you can improve your landing pages and make sure your site is a landing page best practice, putting your best foot forward when visitors arrive. For more best practice information, visit our other blog entries and our newsletter.

For our first Usability 101 blog entry, I will provide a basic understanding of usability testing and address the following questions:

• What is usability testing?
• How important is sample size?
• What are the primary methods for usability testing?
• What are the methodology trade-offs?
• What are the benefits of usability testing?

What is usability testing?

Usability testing is the means for measuring the quality of a user’s experience while interacting with a product or system such as a website, software application, mobile technology, or any user-operated object. Usability testing is best when conducted throughout a product development cycle to capture direct user feedback on the ease of use and satisfaction with the product in order to ensure that it meets users’ needs and business objectives.

Usability testing on websites has evolved from traditional ‘Old School’ product usability testing, which was primarily focused on basic learnability, efficiency, and error prevention, to more advanced ‘New School’ user experience testing, which covers a much broader range of factors that evaluate the quality of the user experience and the impact on calls-to-action that drive business initiatives, as follows:

‘Old School’ Usability

Learnability
Can users quickly and easily learn how to use the site?

Efficiency
How well can users complete primary, routine tasks?

Error Prevention
How frequently do users encounter errors? How can the site design be improved to eliminate common and serious errors?

‘New School’ Usability

User Control
Does the site enable exploration by letting users initiate and control actions based on their primary objectives?

Communication & Relevance
Is the site’s messaging clear and is the value proposition aligned with target users’ objectives?

Aesthetic Integrity
Does the site incorporate graphic elements and visual cues to build a connection and guide users through the site?

Consistency
Does the page layout / organization help orient users to the site experience and set expectations?

Feedback
Are users clearly informed of where they are in the site as they complete tasks? Does the site clearly indicate task progression?

Simplicity & Scannability
Is content on the site presented in a manner that is intuitive and easy to digest based on users’ intent?

Help & Forgiveness
Does the site facilitate the completion of online forms and processes by providing help and forgiveness?

User Satisfaction
Are users satisfied with their experience? How does the online experience impact their future behavior and the future behavior of their friends, relatives, and colleagues?

It is important to consider all of the above factors when conducting website usability testing, as the customer experience is not only about how easy the site is to use, but also about how well the site delivers on customer expectations and drives call-to-action.

How important is sample size?

According to usability veteran, Jakob Nielsen, you only need to test with 5 users to discover 85% of the problems on your site. This is true when the focus of your research is more tactical – focused on learnabilty, efficiency, error prevention – and if you’re testing a fairly homogeneous sample of users.

However, if your study is more strategic – focused on calls-to-action, customer experience benchmarking, making population projections, or measuring the ROI from your online initiatives – it is important to test with a larger sample of users to for strategic decision making and statistically reliable metrics.

In addition, if you’re interested in learning how distinct segments interact with the site (e.g., patients vs. physicians vs. caregivers) or if you’re comparing the needs of different user groups based on demographic or psychographic information, it is necessary to have larger samples to enable segmentation.

Most qualitative (lab-based) research includes 6-8 sessions per day, per market, and can include multiple days of testing as required. Most quantitative (web-based) studies begin at a minimum of 200 participants per study and can range up to 5000+ depending on the breadth information collected, the number of segments, and the type of survey method that is used.

What are the methods for usability testing?

There are two primary methods for usability testing:
1) Lab-based testing
2) Web-based testing

Lab-based usability testing is conducted in person, with a moderator and one or more participants, and can be performed either in-house or in a designated research facility. Typically labs are equipped with audio and video recording capabilities, including picture-in-picture recording for usability testing, and most facilities include a two way mirror to allow clients to discretely view the live interaction between the moderator and the participant(s). Eye tracking monitors can be provided at facilities to enable eye tracking as part of the usability testing.

Web-based usability testing is conducted online, using a browser or proxy-based survey technology that captures the natural behavior of participants as they complete tasks and answer questions online. Participants are invited into a study through an online intercept or email invitation, and they can complete the evaluation in their natural environment, such as their home, office or university.

What are the methodology tradeoffs?

Lab-based usability testing is best when used to uncover low-hanging-fruit problems with user-interface design and to identify clear solutions for resolving those problems. Labs are also helpful when you want to understand the consumer’s emotions, or physical interaction with the site, such as mouse movement or facial expressions. Since usability labs are conducted with a smaller sample (usually 6-8 sessions per day), they are qualitative in nature so the results cannot be projected onto the larger population.

Web-based testing is quantitative in nature and enables larger samples (200+ participants) to complete an online user experience evaluation in a short-timeframe. It also enables more advanced analytics, and statistical significance testing which results in more reliable recommendations and projections. In addition, Web-based testing is completed in the users’ natural setting, from wherever they normally access the Web, so it eliminates any moderator bias or peer influence found in a multi-participant lab or focus group.

What are the benefits of usability testing?

The benefits of Lab-based testing include:
• Smaller sample size provides more qualitative feedback
• Moderated labs enable flexibility in questioning and allow for more in-depth probing
• Direct observation helps showcase participant reactions in person
• Allows for interaction between participants in a focus group or multi-participant lab
• Ability to probe deeper into specific topics or ideas
• Quickly identifies tactical, low-hanging fruit problems and solutions

The benefits of Web-based testing include:
• Larger sample size enables statistically reliable metrics
• Sample mirrors the targeted population (and key segments), including geographic reach
• Users provide honest feedback and behave as they normally would, in a natural setting
• Provides quantitative insights that guide projections and strategic recommendations
• Enables advanced analytics across qualitative, quantitative and behavioral data
• Creates a baseline for future testing and ROI measurement