Keeping up with the competition

Competitive Research offers insights into how to turn new visitors into loyal customers and keep your current customers invested in their relationship with you. It will provide an understanding of the current usability and appeal of your site in direct comparison to your leading competitor(s). By determining what your customers and potential customers think about you and your competition, you can position your site for success, converting new visitors and retaining your loyal customers. In this article, we will describe two Competitive Research methodologies and provide examples of the types of insights that can be gained from this type of evaluation.

Goals and Objectives

Typically, Competitive Research is conducted to meet the following goals:

• Measure site performance against key competitor sites
• Benchmark your site against industry leaders
• Understand major differentiators that drive purchases on your site versus the competition
• Determine which features to showcase to gain a competitive advantage
• Identify short-term and long-term recommendations for site improvements

While Competitive Web Research can be conducted in all industries, the most common are:

• Travel
• Retail
• Mobile
• Financial
• Automotive
• Consumer Products
• Pharmaceutical
• Technology

Detailed Methodology

Target Audience

There are two key target audiences for Competitive Research: your customers and prospective customers. Depending on your research goals, it may make sense to select one audience or the other or a combination of both. The first audience to evaluate is your customers. By testing both your website and your competitor’s with your current customer base, you can understand answers to key questions like:

• What is important to your customers?
• What you are doing that pleases them?
• What are your competitors doing better than you and have the potential to lure your customers away?

Prospective customers are the second audience to evaluate your website, including your competitors’ customers. Testing your site with this audience will provide insights into:

• How do these potential customers see your site?
• What draws their attention and their spending to other sites?
• What would you need to do to entice them to switch to using your site?

A combination of these two audiences may be used to assess both retention and acquisition. A minimum of 100 users per segment is recommended with 200 users viewing each site. This will provide statistically significant results that inform key business decisions with as little margin of error as possible.

Study Design

There are 2 possible designs for Competitive Research: Within Subjects (Sequential Monadic) and Between Subjects (Monadic).

Within Subjects: A Within Subjects design means all of the participants in your study will see all of the sites you are including in your competitive set. This is also known as sequential monadic testing in traditional market research. The benefits of this design include the ability to determine a head-to-head preference and be more cost-effective since it typically requires fewer participants. One drawback to using this methodology is that participants will not be able to perform as many tasks on each site as they might if they were reviewing only one site. This means Within Subjects studies are ideal for tasks that participants are likely to perform on multiple sites in succession, such as shopping for a particular item or researching mobile phone service plans. The design would emulate participants’ natural behavior of moving from one site to another to accomplish a particular task, during which they would be comparing what they found on each site. In this design, participants would be taken to the sites in a random order to mitigate bias and be asked to perform the same task on each site. Following the completion of the task on each site, participants would be asked follow-up questions about their experience on that site before being taken to the next site. After completing the task on all of the sites, generally up to 3 or 4, the participants would be asked about their preference between the sites, on which site they would be most likely to take action, and why.

Between Subjects: In a Between Subjects design, participants are divided between the competitive sites and are asked to perform multiple tasks on a single site with no knowledge of the competitive nature of the study. This is also known as a monadic test design in traditional market research. Participants are asked the same questions on each site after completing the same tasks on each site, allowing for a direct comparison of the data in the analysis phase. This type of competitive research is beneficial if there are 4-5 tasks you would like users to perform on a site - too many to ask participants to complete on multiple sites. Between Subjects is also useful if visiting multiple sites would not be part of the natural process of the behavior being studied. For example, a financial institution may want to evaluate the account management tools they are offering clients against what the competition is doing. It is unlikely that the majority of participants move through multiple banking accounts in sequence looking at these tools. So, in order to replicate the natural experience, it would make more sense to have the bank’s customers evaluate their tools and to recruit customers of the competition to evaluate the performance of the tools in their accounts. The drawback to a Between Subjects design is the inability to ask about a direct preference.

In both designs, detailed questions will be used to probe what participants like about a site, what they do not like, and what would cause them to take action. Both quantitative and qualitative questions in combination with tracked behavioral data will provide a complete picture of how participants are using the sites and the differences in site experience. As this data is collected in the same manner for each of the sites evaluated, the site and brand experience can be compared apples to apples across the competitive set. This provides a clear picture of what is and is not working on your site, as well as the competitors’ sites, informing your strategy to attract and retain customers.

Analysis and Insights

After a Competitive Research project has finished fielding, data analysis begins by comparing the experiences across the various Websites. Differences in the quantitative data are tested for statistical significance and the behavioral and qualitative data are analyzed to understand how participants use each site and the “why’s” of how they feel about each experience. Bringing all this data together for each site evaluated, we can identify what is driving conversion and loyalty on each website to determine where the gaps and opportunities are for your site amongst the competition. Additionally, data can be segmented by current customers and new customers to determine if the groups are looking for different things in a site to help determine how to best serve both of these audiences.

	Analysis - Overall Experience Metrics: How does each site perform on the metrics that are key to a good customer experience on each site? Where does your site outperform others? Where does it underperform? Why? How does it impact how participants would use the sites?
	Analysis - Preference: What sites do participants prefer before their experience? What about afterwards? What sites are they likely to use? What tools / features / content drive preference and response to calls-to-action?
	Analysis - Best Practices: What are the best features of competitive sites? What features and functionality work? What functionality has the competition implemented well? What did the participants dislike?

Based on this analysis, recommendations will be made on what is working well on your site that distinguishes you from the competition and that should be retained, improved or highlighted. Additionally, recommendations will be made to address gaps in content, functionality or branding between your site and those of the competition. Best practices and key features and functionality will be identified and suggestions will be made on how to integrate these into your site. The overall result will be a stronger web presence that takes advantage of your site’s key strengths, acts on opportunities to convert new customers, and retains the loyalty of existing customers - all while establishing you ahead of the competition.

Many clients are concerned with competitive research and already partner with companies like Comscore, Compete or Hitwise. However, these firms offer aggregated data that provides only a cursory view of the competitive landscape. eVOC’s research approach goes deeper than traffic data by directly comparing specific processes and pages to provide actionable recommendations. What is also unique about eVOC’s approach is that you are able to have participants truly interact with the websites as they normally would, rather than answer survey questions after the fact. This interactive surveying technique allows us to understand users’ intentions and combine attitudinal data of what people say with behavioral data of what they actually do. This provides you with a crystal clear picture of how to beat the competition.

Examples of Competitive Research

• Assisted a retail client in improving its checkout process by reducing abandonment and improving the shopping process to better meet the needs of its most loyal customers
• Helped a pharmaceutical company understand how to integrate the content from two merging companies into a single site combining the best of what both companies had to offer
• Helped a travel company determine how to improve its product display pages to increase conversion on its site and reduce the loss of business to competitors

For more information on Competitive Research, download our Competitive Research Overview or contact us with questions!

This entry was posted on Thursday, January 14th, 2010 at 10:01 am and is filed under Methodology. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.