Benchmarking Your Success

One of the most common questions clients ask when we present findings is “How do our ratings compare?” Benchmarks are key for providing context for the customer experience metrics we collect and can offer direction for the prioritization of recommendations and opportunities for innovation. However, determining the right benchmark to use to understand your online customer experience metrics is not as straightforward as it may seem. In a series of blogs about benchmarking, we will explore some of the questions surrounding benchmarks: Who should I benchmark myself against? What metrics should I be comparing? How do I ensure an “apples to apples” comparison? What method should I use for doing so? How often should I be looking at benchmarks?

For this first entry, I want to spend some time thinking about who is your true competition online and, therefore, what companies should you look at when you are thinking about benchmarks?

Part 1: Who are my competitors?

A few years ago, I executed a study that included year-over-year data on a number of travel booking web sites. In the second year, the top site from the first year remained top in the rankings of the competitive set, but showed a significant drop in its year-over-year satisfaction ratings. In exploring what caused the drop, we looked for changes to the site that created barriers to booking or degraded the user experience, but we could find nothing. In fact, the site was exactly the same year-over-year with no major changes made – and that, in and of itself, was enough to cause the site’s scores to fall. No action on the part of the site to stay current with what was happening around it on the internet overall caused the drop.

Looking at only the sites considered the direct competitive set wouldn’t give much insight as this site still topped them all, but the site was losing ground against the best experiences on the Web. So how do you determine who you should be benchmarking your site experience against online?

1. Your direct competition is of top importance. Understanding what is happening in your industry and how you rate against those competitors still provides the most useful context for understanding your metrics. For example, determining if features and functionality are becoming standard in your industry, identifying items missing from your site and those areas in which you excel. This can be done most easily through a head-to-head online evaluation.

2. Compare your site’s critical functionality to innovators of that functionality in other industries. Consider the functionality of your Website and look to the innovators in that specific functionality for a benchmark and best practices. For example, if you are incorporating video on your site, it is important to understand how it impacts your position among your direct competitive set. Users are not just looking at the video implementations on your competitors’ sites and forming an opinion based on those implementations, but they are also looking at YouTube.com, iTunes and Hulu.com, just to name a few, that have now become your competitors in this area. The usability and functionality of these sites are what users will be comparing you to, and therefore are what you need to be benchmarking your video implementation against. Just like in the offline world, a travel company may look to Dell as a leader in customer service, you need to determine who users will be comparing the different elements of your site to and look to those sites for benchmarks to measure yourself against and best practices to help improve your implementation. When considering new functionality, be sure you understand the benchmark set by these sites and use those benchmarks as goals for your design – measuring your design against them once it is complete.

3. Study averages across all industries. Because users’ opinions of the experience on your site are driven by all of their Web experiences, looking at customer experience metrics from a wide variety of sites in aggregate can provide important context. In the Benchmarks area of our site, we show benchmarks from hundreds of evaluations across industries and types of sites. This data is also the data we incorporate into our presentations most often because it takes into account the fact that users do not only judge a retail site by the experience they have on Target.com, Amazon.com and BestBuy.com, but also Google.com and Match.com. When it comes to usability and customer experience, users’ interactions on one Website form their expectations for another across industries, content-types and functionalities. Aggregated figures allow you to understand where your site fits among all the experiences a user may have online giving you an accurate idea for where your site is ranking.

By utilizing a combination of these three “competitive” sets for benchmarking depending on the situation and objectives of your research, you will be able to gain a complete picture of the customer experience on your site and continue to meet and exceed customer expectations by ensuring that you have a complete understanding of the playing field that is shaping those expectations.

Stay tuned for Part 2 – what metrics should you be benchmarking?