In my recent article about intuitive design I questioned why so many companies produce products and services that under-perform relative to customer expectations. There I concluded that companies are pressured to minimize development time in order to gain early market entry (or catch up quickly) and return shareholder equity. While those are valid points, my current investigation into conjoint analysis and choice models points to another crucial element: the complexities of consumer choice make critical success factors extremely difficult to predict.
To put this in perspective, consider the data offered by Edison Innovations:
- Nearly 98% of all patents fail to break even and only 56% of new product introductions achieve their financial goals.
- Nearly 90% of consumer products fail.
- They fail not because they don’t function, but often because they do not address a big enough consumer need at a price people will be eager to pay.
- They were the wrong products to develop — or the right product, delivered to the wrong customer or positioned to target the wrong need.
Why so many failures? It’s all about context. Simply asking people to rate their preferences from a static list excludes the reality that there are trade-offs. Results from this survey method are not predictable enough to ensure success. Think about it this way: if I go to Home Depot to purchase a hose (something that’s pretty simple, right) I’m going to evaluate color, length, construction and price. This gets sticky when there are multiple options for each factor. I could get a 25′, 50′, 75′, 100′, 150′ or 200′ hose length in green or black, built with 1-ply, 2-ply or 3-ply construction for $7 to $45.
If I don’t see exactly what I’m looking for I’m going to have to make some compromises. Since the act of compromising is likely to diminish my brand loyalty, the goal of every manufacturer should be to know exactly what I’m looking for before they build the product. Ok, well maybe not me in particular, but they should have measured a wide enough sample to understand how these factors will affect buying decisions.
“I was told there’d be no math”
In a classic Saturday Night Live skit Chevy Chase plays presidential candidate Gerald Ford. During the mock debate he is stymied by a question about the federal deficit. Sweating and twitching uncomfortably, he responds with the now-famous punch line “I was told there’d be no math.”
If you’ve ever felt this way, don’t worry. I’m about to introduce some complicated terms but we’ll stay on track with the business explanation, and won’t have to re-experience the pain you may have experienced during statistics, algebra or calculus classes. We only need to understand that there is a relatively new level of pre-development metrics available to Product Managers. To really get where they are coming from we need to understand two key concepts, which are evolving to a new level:
Along comes the web:
These methods have been around for a while. The thing that really turbo-charges their application is the rich development and propagation of the internet. About a decade ago researchers determined that there is very little difference in a customer’s choice pattern between actual and/or perceived products. In other words, we don’t really prefer an actual product to a ‘virtual’ product when feature shopping. Hence the overwhelming success of e-commerce.
From a producer’s perspective, it is much more desirable to build ‘virtual’ prototypes than actual live models. The cost, time and potential market share differences are obvious. Besides, the ‘virtual’ development environment allows analysis to scale well beyond physical limitations. We are talking about an order of magnitude here. All of a sudden, with advanced algorithm based SaaS applications, manufacturers can access a level of analysis previously deemed impossible.
A paper by Ely Dahan (Sloan) and V. Srinivasan (Stanford) (1998) sums up the development well:
“The Web can help to reduce the uncertainty and cost of new product introductions by allowing more ideas to be concept tested in parallel. It’s about the need to consider many product concepts since only a small percentage of new product ideas ultimately prove to be profitable. Keeping multiple product concept options open and freezing the concept late in the development process affords the flexibility to respond to market and technology shifts and may actually shorten total product development time. In short, there is a pressing need for low-cost, parallel testing of new product concepts.”
What this means to your company:
Traditionally, “market research” companies have focused on data building and portfolio selling. They build sample sets, poll the participants and report the results. Many would like us to consider them as ‘full service’ partners who are capable of measuring customer satisfaction, retention, win-back and brand loyalty while assisting with our brand positioning, market segmentation, etc.
My brief research leads me to believe that the industry in general is still using standard ‘questionnaire’ profiling systems (mail, telephone & online) that lack advanced algorithms, and that their services are therefore far more linear (and limited) in nature than we might presume.
One company (Affinnova from Waltham, MA) is oriented around real-time analysis used to refine the data sets and propose solutions. I wanted to understand why Affinnova is positioning itself as a technology company while most of their competitors bill themselves as market research organizations. Yahoo’s business profile describes the value proposition succinctly:
“The company provides software that helps consumer products companies develop and package products that will be most appealing to consumers. Affinnova’s software enables companies to test out and adjust product concepts and designs before they actually get to the shelf. The software creates a realistic graphical prototype of a product, packaging concept, or advertisement, then it collects consumer responses and analyzes them in order to determine which variations consumers prefer.”
Referring back to the product failure statistics listed above, how valuable do you think this kind of pre-production information could be to your business? To round things out a bit I wanted to investigate some of Affinnova’s competitors. Their Yahoo profile lists Greenfield Online, Harris Interactive and MarketTool as rivals. Following up on each of them, I decided to also add Lightspeed Online Research and Survey Sampling International to the mix.
Companies to consider when you are prototyping:
Here’s a synopsis of each (partially borrowed from Yahoo Finance profiles):
- Greenfield Online, Inc.:
A survey community that harvests marketing data. The company provides survey research services, gathering demographic data over the Web from its panel of more than 1 million people. It performs surveys to track brand awareness and test advertising campaigns for its clients, as well as to measure consumer interest in new products and services. Its Ciao subsidiary provides comparison shopping services for consumer products, primarily to the European market. The company was acquired by Microsoft for $486 million in October 2008.
- Harris Interactive Inc.:
The company is a leading market research and opinion polling firm best known for its Harris Poll, which takes the pulse of public opinion on various topics each week. Harris conducts its market research and polling primarily through an Internet panel consisting of millions of people from around the world. #13th largest market research firm in the world. Annual revenue = $237.8 m (2008)
- Lightspeed Online Research Inc.:
Lightspeed gathers its data through online panels and also provides custom research for clients. Throughout its paneling networks, the company can access nearly seven million household members across more than 30 countries in the Asia/Pacific region, Europe, and North America. Established in 2000, Lightspeed conducts a variety of studies including customer satisfaction and retention, attitude and awareness tests, Web site evaluations, new product and concept tests, brand awareness, and in-home product tests.
Their research is segmented into a main global panel, as well as vertical alignments (Financial Services Group, Employee Surveys, Custom/Specialty Panels and Mobile Surveys). Annual revenue = $55m (2007)
- Survey Sampling International LLC:
The company’s sample services help clients minimize research risk, reduce costs, and improve research quality by identifying the best subjects for a survey. SSI can screen potential respondents by age, sex, ethnic background, income, and other demographic criteria as well as interests and lifestyle. It also provides highly defined samples for business surveys. SSI has more than 1,800 clients worldwide (including nearly three quarters of the top research firms) that use its services for Internet, telephone, and mail polls and surveys.
SSI provides access to more than 6 million research respondents in 54 countries and does seem to focus on access rather exclusively, not necessarily orienting around the results or interpretation of acquired data.
- MarketTools, Inc.:
MarketTools gives businesses a peak inside their customers’ heads. The company provides online market research services and technology, helping clients with advertising research, concept testing, and product development. Among the company’s offerings are Zoomerang, an online self-service survey; TrueSample, a managed target audience panel; and zTelligence, a technology platform for capturing and reporting survey data.
If you are responsible for product development, you want to understand how these kinds of companies operate. In general, there are some basic common factors. Each will utilize:
-
Panels & Communities (with an emphasis on data integrity).
Concept testing.
Analysis.
Modification.
A final note about the opportunities here:
I wonder, could advanced predictability tools be used to proactively evaluate existing product categories, measure the inherent strengths and weaknesses, and model new solutions that companies might not even be considering yet? Could a company like Affinnova, for example, research various product/service categories to determine the ideal consumer mix, then find a way to monetize that information instead of waiting to do project-based work for specific principles?
Perhaps you noted above that Microsoft paid $486 million last fall for Greenfield Online. To put this in perspective, Harris Interactive (the 13th largest Market Research company in the world) has annual revenues of $237 million. Clearly the folks from Seattle think this gamble is going to pay off big time in the future.
Do you think they’ll use the information to develop their own products? If so I’ve got news for you. But that will have to wait until my next blog. (Hint: Greenfield hasn’t figured out the customer experience yet).
[...] Product Development: Conjoint Analysis & Choice Models Hold the Keys [...]