Posts Tagged ‘mobile usability’

Testing Desktop & Mobile in a Single Usability Session

May 8th, 2013
by sabrina.shimada

In the past year we have seen an increasing demand for multi-device usability testing. Multi-device usability testing involves asking users to perform tasks on more than one device in a single session; devices may include desktop computers, tablets, and mobile phones. Testing multiple devices in a single session has been useful when testing Responsive Web Designs, as well as testing the fluidity of a brand’s experience from desktop websites to mobile and tablet websites, and/or mobile apps.

One benefit to conducting testing across devices in one session vs. testing designs on a single device per session is that the user is able to shed light on the overall experience and consistency across devices. How fluid is the experience? How easy is it for the user to transition from desktop to mobile and vice versa? Not only will testing multiple devices with one user provide insights into an overall experience, but it can also give direction and insight into how and why users may gravitate to one device for certain tasks vs. another.

What we have learned from conducting these tests is that preparation and flexibility play key roles in making sure this type of testing goes smoothly. Below are a few tips to help you prepare for and conduct multi-device usability testing:


  • Recruit to reflect your real audience: If 90% of those using your mobile app are iPhone users, then testing should focus on iPhones vs. recruiting an even mix of iPhone and Android users.
  • Forget the Gadget Lab: We have seen greater success in this type of testing (and mobile-only testing) by having the user bring in their own device. Not only is the user more confident navigating the device, but often there are added insights. With the users’ own device, you may get to see what apps the user has downloaded and how they have organized the information on their phone. Just be sure to clearly specify the types of mobile devices, model and version of software / operating system (e.g., FROYO, GINGERBREAD, etc.) required for testing during the screening process.
  • Get the OK to Download: If users will need to download anything on their phone or tablet, it is best to ask this during the screening process to avoid any trouble down the road.

Discussion Guide

  • Counterbalance Starting Device: Typically we will start an equal number of participants on each device (e.g., N=12 / 6 start on desktop, 6 start on mobile), unless the team is focused on the usability of one more than the other. We also make sure that we have an even mix of participant types (if research has segments) for each starting device. For example, if we have a total of 12 participants, with 6 Prospects and 6 Current Users, then 3 Prospects and 3 Current Users will start on desktop and 3 Prospects and 3 Current Users will start on mobile.
  • Repeating Tasks is OK: It is okay to repeat tasks across devices; we have found that usability issues are still uncovered. Users will have expectations based on their initial experience, but this happens in real life, too! It is nice to add some variety to the tasks if possible, but do not stress if prototypes only allow for the same tasks to be performed on the various devices. By shuffling the starting device, the team will still be able to get initial impressions on the experience from a portion of the users. In fact, if you are looking for an apples-to-apples comparison across devices, it is important that you have the user perform the exact same task on each device.

Set Up

  • Wi-Fi Ready: We typically ask users to join the Wi-Fi network on their phones or tablet prior to starting the session to save time. If you are conducting labs at a facility, it’s helpful to ask the front desk to share the Wi-Fi information to the participant upon arrival.
  • Bring a Back-Up Device: Yes, we did say to forget the gadget lab; however, it does not hurt to have a back-up device for unpredictable technical difficulties. Typically, we just bring in the most popular mobile / tablet device used to access the site/app in the case that the user’s device fails to work during the test.
  • Device Hot Spot: To ensure that the respondents’ actions on their mobile device are captured, it is best to designate an area for the camera to target. Typically, we tape off the area where the users should keep their device in order for it to be successfully captured on camera.
  • Capture All Angles: Many of our clients view sessions remotely and it is a much more engaging experience if all angles of testing are captured, which includes the respondent’s face at all times and then the desktop screen, mobile phone, or tablet. In order to do this, we typically have multiple cameras in the room, which can be controlled by our video technician in the backroom. Our video technician is on site during the sessions to switch between cameras during testing to make sure all angles are captured at the appropriate times.

Conducting the Usability Session

  • Keep it Real: One thing to avoid in user testing is forcing the participant to complete tasks on a device he or she is not familiar with or would not use in real life.
  • Ask Wrap-Up Questions: One of the biggest benefits to multi-device usability testing is the ability to understand the overall experience across devices, so don’t forget to ask questions about this! Some questions you might ask are, “How does the desktop experience compare to the mobile experience?” or “Are there any tasks you prefer to do on one device vs. another?”

That sums up our tips for preparing and conducting multi-device usability testing. As each research project is unique, there are always ways to refine and adjust the methodology to ensure that the research objectives for your project will be met. If you have any questions, please do not hesitate to reach out to us at

  • Share/Bookmark

Mobile Website Usability Best Practices - Mobile Websites

March 5th, 2012
by sabrina.shimada

The mobile web is unique; users do not exhibit the same behavior on their mobile phone as they do on their desktop computer. This blog series will point out some of the best practices of mobile website usability by looking at various mobile websites.

Why Your Mobile Strategy Should Include a Mobile Website

Fairly recently, Nielson Norman Group’s usability guru, Jakob Nielson, came out with research findings that tout mobile apps as having better usability than mobile websites, urging companies to go the app route. Nielson pads these findings with a prediction that mobile websites are not too far behind, but his closing advice reads, “Today, if you are serious about creating the best possible mobile user experience, my advice is to develop apps.”

I disagree and here’s why (the abbreviated version).

First step in a positive user experience is providing an experience.

Out of 1.2 billion mobile users, there is a 2:1 ratio for feature phones to smart phones. If you are trying to reach a global audience, this means you are leaving out a huge group of prospective customers. In the US, the majority of web traffic is already dominated by smartphones, but 2.4% of non-computer traffic still comes from feature phones. If you can narrow your users down to smartphones then the second issue you have with mobile apps is optimizing for various devices, your app needs to be optimized for Androids (35.6% non-computer traffic), iPhones (23.5% non-computer traffic), and other smartphones (6.7% non-computer traffic) according to comScore.

Fact: Facebook sees more people accessing their mobile site than from their top native apps combined.

What if users don’t want to download your app?

I know Nielson tested the usability of the apps, but did they do any open-mobile web research? Meaning, did they research how users arrive to mobile sites and applications and test how users react when they hit a page that prompts them to download an app? No, and the way I see it is that any extra step, click, hop, or jump you make the user go through increases the likelihood for drop-off. From personal experience, I have searched and navigated my way to websites that prompt me to download their mobile app, and what do I do? I pass. If there is no direct link to their website, I will hit the back button on my browser and choose another site from my Google search results or I will just take a look at the site later on my computer…that is if I remember.

While there is no research that delves specifically into site abandonment when prompted with an app download, MobiThinking reported that “In all countries surveyed more consumers used their browser than apps and only a minority will use web or apps exclusively. US consumers prefer mobile browsers for banking, travel, shopping, local info, news, video, sports and blogs and prefer apps for games, social media, maps and music.” I am not surprised by this and by no means would I advise a gaming company to dive straight into a mobile website at this stage, considering HTML5 performance (arguably) still lags behind native apps.

Lastly, who says mobile sites can’t offer a great customer experience?

This blog series is going to showcase mobile sites and point out some of the do’s and don’ts as we see them; however, I’d like to start with three do’s that apply to all mobile sites.

  1. Recognition - Your site needs to recognize mobile devices
  2. Speed - Site should load and navigate through pages/searches quickly
  3. Opt-Out Option - Users should be able to easily find a link to full site
  4. Limit the Work - Site should be organized and easy to use as to prevent users from having to search, scroll, click or type too much
  5. Size it Up - Font sizes and buttons should be legible and sized to appropriate tap zones according to type of mobile browser

Now, let’s take a look at our first example of a mobile site and point out the Do’s and Don’ts!

Example 1: AirBnB - Finding a Room Nearby


AirBnB’s initial page wastes no time in presenting users with lodging options by showing nearby locations right from the get-go. Whereas Expedia offers users to ‘Search Nearby Tonight’ or geo-locate via compass icon next to the destination field, both of which require the user to tap another button and load a new screen. Not to mention, when I tapped the ‘Search Nearby Tonight’ I got the error message seen in the pic below which was not informative or helpful at all. Don’t waste users’ time with such messages, make the operation work or come up with better error messaging.

Expedia does, however, offer sorting features not available on the AirBnB site which are extremely helpful when searching for a place to stay. Users can search by price, popularity, or neighborhood, among other options.

To sum it up, the site has minor usability flaws, but overall is an easy to use mobile site that allows users to quickly search and find a room. All of the call-to-action buttons are clearly labeled and appear above the fold. During my site experience there were no loading issues or major roadblocks. When I checked out a few competitor sites, I was surprised to see that few had mobile websites (Roomarama, Wimdu) or if they did, the websites had inferior user experience right off the bat (VRBO, Couchsurfing). Ultimately, AirBnB gets 4/5 stars for overall satisfaction and ease of use in my book.
Stay tuned as I review more mobile websites across industries. If you have any suggestions for sites we should look at, catch us on twitter @evocinsights.

  • Share/Bookmark

Usability Week Recap: Day 2 - Mobile User Experience

June 29th, 2011
by sabrina.shimada

USABILITY WEEK 2011 Continued…
Last week, eVOC attended Usability Week 2011, a conference organized by the UX consulting and research firm Nielsen Norman Group (NNG). My colleague Phil and I split up to tackle full day tutorials on Mobile User Experience. Click here to read his post about Day 1 – ‘Mobile User Experience 1: Usability of Websites and Apps on Mobile Devices’.

My agenda for Day 2 included four cups of coffee and a full day tutorial titled ‘Mobile User Experience 2: Touchscreen Application Usability,’ led by Raluca Budiu, a User Experience Specialist with Nielsen Norman Group. Besides Raluca, it seemed there were very few usability or customer experience professionals in the room, but mostly designers and developers hoping to get some direction on how to design for the mobile customer experience.

My 2 Cents

Overall, day 2 focused on the do’s and do not’s of mobile design. I was impressed with how many aspects of mobile design were covered, but can’t say I was surprised by many of the points made. Here I’d like to share with you 10 of the most important mobile best practices (in my opinion) pulled from the 147 provided by Nielsen Norman Group, along with my own 2 cents.

1.  “Detect if the user is coming from a mobile phone and redirect him or her.”

  • Put your best foot forward, why make users struggle with your regular site if you have a mobile one? That being said, it should be easy for the user to navigate from your mobile site to your regular site, vice versa.

    2.  “Keep launch time to a minimum.”

    • There are hundreds of apps and endless ways to kill time on a mobile phone. If your mobile site or application takes forever to launch, what makes you think users will wait? The user may be intrigued enough to wait the first time they use the site or application, but after learning that it takes too long (for some it’s seconds) they may never return.  I know I’ve ditched slow launching apps, what about you?

      3.  “Respect platform conventions.”

      • Throughout the tutorial, Raluca asked the audience to analyze various mobile sites and applications, asking us to identify the good and the bad aspects of the different designs. I found that many designs for the iPad failed to take advantage of its screen space, so make sure you respect platform conventions!

        4.  “Avoid requiring users to login or register unless absolutely necessary.”

        • This is an issue of ease and time. If you are going to require a log-in, there better be a point. Is there valuable information that needs to be saved in the users’ account? Or are you just trying to build a mailing list?

          5.  “Minimize user input by using techniques such as autocomplete, suggestions, sensible defaults (based on user history or current context).”

          • Using a small keypad or touchscreen is not as easy as typing on a keyboard, make life easier for your users by reading their minds! Instead of requiring lots of typing, I think giving suggestions or providing options that require a simple tap can make or break its usability.

          6.  “Allow saving and sharing of content seen during session (e.g., email, Facebook, wish lists).

          • Personally, I find myself finishing most of productive activity that I started on my phone on my laptop. Being able to save drafts, lists, etc. to access at a later time or even on the regular website is beneficial to users.

              7.  “Make sure that all content is available both in landscape and portrait mode.”

              • Remember, mobile devices are MOBILE, they move, thus your designs should work with the user and be able to move with the direction of the screen. This means portrait and landscape for phones and all four orientations for the iPad.

                8.  “Make sure users can easily turn notifications on/off.”

                • Phones ring, vibrate, and ping enough – don’t bother your users.

                  9.  “Users should be able to use your app without reading instructions or a manual.”

                  • If you want users to use your app, make it usable. It’s that simple.

                  10.  “Save state within app, so that if the user returns to a previously visited tab they see the same data they were looking at last time when they visited the tab.”

                  • This is vital when it comes to mobile devices because users are constantly going back and forth between using their device as well as hopping from app to app. It’s important to remember how mobile devices are used differently than computers – mobile users are often on the move vs. sitting down.

                    As always with best practices, there are general rules and then rules that are specific to your industry, customer base, and media platform. If you are interested in testing the usability of your site or application on a mobile device, please visit our page on usability testing for mobile devices and don’t hesitate to contact us to find out more.

                    • Share/Bookmark

                    Usability Week Recap: Day 1 - Mobile User Experience

                    June 24th, 2011
                    by Phil Scarampi

                    Last week, eVOC attended Usability Week 2011, a conference organized by the UX consulting and research firm Nielsen Norman Group (NNG). On Friday, I attended a full-day tutorial taught by NNG’s User Experience Specialist Raluca Budiu, entitled ‘Mobile User Experience 1: Usability of Websites and Apps on Mobile Devices’ (view the full itinerary here).

                    The purpose of the seminar was to share tips on developing a mobile strategy, outline the differences between building mobile websites vs. applications, and reveal best practices across these different media. NNG pulled data from their own international diary study, user testing, and design reviews, as well as from various research articles.

                    My colleague Sabrina and I tag-teamed the event. And by tag-teamed, I mean I got to go on Friday, and she got to go on Saturday. Though she’ll be taking a day off any time she likes, so she may be having the last laugh. In any case, stay tuned for her upcoming blog post about her day at the event.

                    Overall, I found the day I attended to be relevant and engaging. Raluca provided a lot of interesting insights about the mobile user experience. Below are some of my observations:

                    It is truly amazing how so many of the common conventions we follow for websites on a computer apply equally to mobile devices. Things like messaging, layout, navigation, error messaging, and spacing are all crucial in developing a usable interface for users to get things done – regardless of medium. I’d say 80% of the best practices discussed would apply to every type of device on the market.

                    Mobile is still such a new medium that many businesses do not have the experience or knowledge to develop a cohesive mobile strategy. At Usability Week, the majority of attendees were designers or programmers at companies just starting to roll out a mobile platform. Fortunately, they can model themselves after other businesses across industries that are well established in mobile, such as Amazon, Google, and Walmart. But despite the efforts of these trendsetters, there is still opportunity for great innovation as technology improves and social media grows.

                    Something I knew but had never verbalized: There are two reasons people access data on their mobile devices. One is to pass the time by browsing. The other is to search for a specific piece of simple information (people generally look for more complex information on their desktop/laptop computers). It helps to know this so that a site can be tailored to these two reasons.

                    Something I know and have verbalized: Repeat visitors are more likely to use mobile apps, while casual/occasional visitors are more likely to use mobile websites. It’s therefore no surprise that apps have higher success rates than mobile sites. The reasons for this are twofold: 1) repeat visitors are more likely to use apps and have an easier time completing tasks due to their familiarity with the site. 2) Mobile sites tend to have more capabilities than mobile apps, making them harder to navigate and complete tasks.

                    Something I didn’t know: Apps put less strain on infrastructure because there are fewer server calls to load content.

                    Something that really surprised me: Some companies don’t automatically recognize when a mobile user lands on their website. These visitors are forced to browse the full site on their mobile device and consistently have trouble.

                    One of NNG’s findings is that companies should avoid designing for the lowest common denominator. Their belief is that by trying to satisfy everyone, you most likely satisfy no one. This often goes counter to what companies want to do on their websites. Even if just 2% of visitors are using a certain media to access your site, it’s natural to want to account for them. However, if designing a good experience for that 2% means deteriorating the experience of the 10% who use a different, more ubiquitous media, it is often a better business decision to favor the 10%. And as with all business decisions, key factors must be considered, such as audience, content, budget, priorities, and needs.

                    Finally, NNG did not spend a lot of time discussing how to conduct mobile testing. But a theme I noticed throughout the day was just how much of their insights and knowledge came from conducting in-person mobile usability testing.

                    Above all, it’s crucial to know your audience and what they want to do on your app or site. We conduct our own mobile testing that helps companies understand their customers, improve their usability, and increase ROI. Click here to learn more.

                    • Share/Bookmark

                    Our Services

                    Web Based Testing

                    Website Evaluation

                    Competitive Assessment

                    Intercept Survey

                    Prototype Testing

                    Home Page Survey

                    Brand & Concept Test

                    Open Web Research

                    Online Benchmarking

                    Consulting Services

                    Expert UI-Review

                    UI Consulting

                    Lab Based Testing

                    Usability Testing

                    Focus Groups

                    Eye Tracking

                    In-Depth Interviews

                    Card Sorting

                    Persona Research

                    Featured Articles

                    Usability 101

                    Methodology Spotlight

                    E-commerce Usability

                    Website Usability Testing

                    Our Expertise



                    Case Studies

                    About Us

                    Management Team


                    News & Events

                    Contact Us



                    Connect With Us

                    linkedin logo

                    Affiliations and Endorsements

                    evoc greenbook badge Featured Market Research Companies

                    Keep Informed

                    Sign up for eVOC's monthly newsletter.