When it comes to data, many multi-channel retail businesses face an embarrassment of riches. What’s missing is timely, actionable insight. Data mining is the centerpiece of an analytics strategy that can deliver business value.
Understanding customer behavior is important to adjusting business strategies, increasing revenues, and identifying new opportunities. The vital importance of such knowledge to these objectives isn’t new; in fact, it’s always been a fixture of business success. What’s new is that many organizations now have at their disposal an impressive variety of data and information resources that promise to reveal far more about customer behavior than previously thought possible. This potential has unfortunately created an agonizing paradox, especially for large and diverse businesses: the more data resources available, the harder it is for the organization to understand its customers.
Today, detailed customer interaction data is abundant. We have data about browsing behavior, purchase behavior, returns, complaints, wishes, gifts, and more. Yet, how many businesses are truly using this data effectively? The reason for this paradox is that technology for generating, capturing, and storing data has far outpaced the human capacity to understand, analyze, and exploit it for maximum impact. Data mining technology, which focuses on identifying interesting patterns and developing predictive models from data, has the greatest potential for enabling businesses to leverage data resources for strategic business success.
THE RETAIL CHALLENGE
Nowhere is the paradox of rich data and poor utilization more apparent then in a multichannel retail environment. Initial data-mining efforts in the online domain focused primarily on site statistics and transaction logs, analyzing such factors as how many hits a site received, which pages visitors viewed, how long customers stayed at the site, what they purchased, and how much they paid. Such Web site analytics are fine as aggregate statistics, but they don’t go nearly far enough to help retailers get the most value out of customer interactions.
To obtain a truly comprehensive view of how customers interact with your store, you also need to combine online data with data from other sources, such as demographic information and records of in-store and catalog purchases. You must also be able to segment customers according to a variety of criteria, and then analyze the specific behaviors of each segment. Finally, you need an efficient means of turning these insights into action – for example, creating promotions, campaigns, and “related items” recommendations that target particular groups of customers. With such a comprehensive set of capabilities, you can realize the ultimate benefit of data mining: gaining in-depth customer insights and acting on them to increase customers’ purchases – and revenues for your business.
The first step I’ll address in this article is to consider the different types of customer data that can be gathered and the various stages that retailers must go through to learn how to take better advantage of data mining technology. Next, I’ll describe how you can harness advanced analytics to optimize customer interactions and improve your bottom line. Finally, I’ll discuss what’s required to take full advantage of the insights gained from data mining.
WHAT DATA DO YOU NEED?
For most retail environments today, three sources of customer data are most critical to data mining efforts toward better understanding of behavior.
Demographic data. Direct marketers have employed data about age, geographical location, and income for many years to target specific groups of customers. The goal has been to use this data to aim promotional campaigns at groups with particular interests.
Transaction data. This resource provides concrete data about what your customers are purchasing. Going beyond general demographic information, transaction data is essential in helping you predict future purchase and target promotional campaigns more effectively. In addition to the value of the transaction itself, this data also reveals key information about time, location, and other factors related to the transaction.
Online interaction data. The dominant form of data is Internet clickstream data, although we must also include interactions that occur through wireless devices, cable television, and more. This resource can provide deeper information than transaction data; it provides a window on customers’ decision processes and the navigational steps they took to find what they desired. Online interaction data records every page that the customer saw leading up to a decision. Therefore, you know not only what they purchased (or didn’t purchase), but you also have strong evidence of how they arrived at that decision.
By better understanding the process leading up to the buy decision, you can more effectively influence future purchase decisions. For instance, suppose you feature a particular product on your home page. However, when customers click through to look at it, they end up buying a different product that they found through a link from the featured product’s page. That tells you you’re featuring the wrong product on your home page. Or, it might confirm that the featured product is indeed a good candidate for a “loss leader.”
Equally important, clickstream data tells you who didn’t buy your products and why. For example, clickstream data might show that of the customers who left the site without making a purchase, many were searching for shipping information just before they dropped out. You can conclude that you need to make shipping information clearer and more readily available – and you could test this theory and see if higher sales and fewer abandoned shopping carts result.
RETAIL USES OF DATA MINING
While most online retailers today gather some sort of statistics on the efficacy of their Web site, the majority haven’t begun to tap the fill potential of data mining. Retailers typically progress through three stages as they come to understand how in-depth mining of customer interaction data can help them meet customer needs and increase profits.
Stage 1. Web analytics consist of gathering Web site statistics that track customers’ online behavior; how many hits your site gets, how many pages customers view, the dollar volume of sales, and so forth. This type of feedback can be helpful both for fine-tuning your Web site to better meet your customers’ needs and for identifying such factors as which of your products and services generate the highest (or lowest) online revenues.
The problem is that Web site statistics only help you analyze one aspect of customer interaction: their online behavior. The statistics don’t capture transactions made through catalog sales or bricks-and-mortar stores; they don’t include customer demographics; and they don’t provide any way to segment customers. So, while Web site statistics are a good stat, they only scratch the surface of the benefits that advanced data-mining techniques can bring to your business. Borrowing the phraseology of Geoffrey Moore’s Technology Adoption Model (see Resources, page 26), the “late majority” of retailers stop at this stage.
Stage 2. Customer analytics adds depth to understanding customer interactions. This stage, now becoming mainstream within the “early majority: of retail adopters, is where companies gather data from multiple sources, including Web site interactions, transaction data from offline purchases, and demographic data from customer registration forms. A good customer analytics solution bases analyses not on data subsets or high-level aggregations, but on every individual transaction. This approach brings both higher accuracy and the ability to drill down to more detailed views of how your customers interact with your company. The richness of this stage of analytics offers a more holistic view of your customer, with deeper insights into their behaviors, likes, and dislikes.
Good stage 2 customer analytics solutions also include the ability to segment customers according to a variety of criteria and export the results to other programs. That way, as you learn more about the behavior of a particular subset of your customers – say, high-volume purchasers in the 30-to-50-year old age group – you can target that group with specific marketing campaigns. Such campaigns tend to produce much better results than more general approaches and are also less likely to annoy customers who aren’t interested in what you offer.
Stage 3. Optimization, adopted so far only by the retail visionaries, is the most advanced stage of data mining usage and offers the biggest potential payoff. In this stage, sophisticated data-mining algorithms sift through data volumes to discover patters that may be too subtle for humans to distinguish. The software applications can then automatically apply the insights to optimize customer interactions. In other words, by tailoring recommendations and promotions to the preferences of specific customer groups, you can actually change customer behavior: upselling them to a higher-priced product; crosselling to additional, related products; or even downselling to a lower-priced product in cases where the customer is abandoning a potential purchase because its price is too high. These recommendations, based on data patters, can produce immediate payoffs in the form of increased sales. Because the marginal cost of these incremental sales are minimal, the contribution to the profit margin can be dramatic since most of the costs associated with each customer have been sunk into the primary effort of driving them to your site or store.
Retailers can derive smarter recommendations through data mining. Recommendations are generally made first to all customers; then to specific segments of customers; and finally to individual customers on a one-to-one basis reflecting knowledge of their preferences. As an example of the first type of recommendation, the data might reveal a connection between customers who buy backpacks and those that buy jeans. The Web site could then display a jeans promotion or link whenever any customer place backpacks in their shopping carts.
More data mining could then allow the retailer to pursue the second type of recommendation. Continuing our example, the data might reveal that customer who buy a particular type of backpack prefer a specific style of jeans. The retailer could arrange the Web site to have a link between these exact styles. Finally, further data mining could enable the third type of recommendation, allowing the retailer to know that particular customers prefer Diesel jeans, and showcase Diesel jeans whenever the site displays a jeans link to that particular customer.
Ultimately, retailers can use data knowledge to make recommendations across sales channels. Multi-channel customers are known to have a higher lifetime value that single-channel customers. By marketing to them more effectively, you can further increase their value. For example, some multi-channel customers ma prefer to use your Web site to view products, but then go to one of your stores to purchase the product )for example, to verify that an item fits). With multi-channel customer data analysis, you can discover such a pattern and then bring that knowledge to bear by creating more effective marketing campaigns: for example, by emailing those customers a coupon that they can redeem at a physical store. Conversely, you could leverage knowledge of multi-channel customers’ offline purchases to tailor promotions that enhance their online shopping.
OPTIMIZATION ANALYTICS: PUTTING BUSINESS FIRST
We should never forget that data-driven applications and data mining technology shouldn’t drive customer interactions. Business rules should govern much of the use of data-driven technology. A data mining prediction that confidently tells us that a particular customer will buy a particular item is of little use when that item is no longer in stock.
Furthermore, business rules need to take priority when it comes to branding, strategy, and relationship management. Supplier constraints might also apply: For example, you might specify that the site link purchasers of Kenneth Cole items only to other Kenneth Cole items; or that that site link purchases of jewelry only to other jewelry, perhaps in order to keep the focus on higher-priced items. Optimization analytics can provide you with regular reports that show the effectiveness of such automatic recommendations. Bottom line: It’s crucial that the technology you use for data-driven applications allows for and understands business rules and constraints.
J. Crew, a major online and catalog retailer of men’s and women’s apparel, shows, and accessories, has had notable success with optimization analytics. The company previously used a cumbersome manual procedure to recommend similar and complementary styles to online purchasers. In the fall of 2002, the company moved up to optimization analytics. They found that automatically generated recommendations generated twice as many sales as the manually generated ones.
For another leading specialty fashion retailers, optimization analytics enabled the company to improve the effectiveness of “related items” merchandising, where the retailer suggests items to shoppers as they browse the Web site. After setting up the optimization analytics system, the retailer was able to improve monthly sales revenue results by more than 40 percent above the results gained by previous, manual efforts. The analytics also saved the merchandising team more than 100 hours of labor each month.
USAMA FAYYAD
Is CEO, Chairman, and cofounder of digiMine, a provider of customer interaction intelligence services. Prior to digiMine, he founded and led Microsoft Research’s Data Mining & Exploration Group from 1995 to 2000. From 1989 to 1995, he founded the Machine Learning Systems Group and developed data mining systems for the analysis of large scientific databases at the Jet Propulsion Laboratory, California Institute of Technology. He is a director of the Association of Computing Machinery SIGKDD and serves as editor-in-chief of its newsletter.
ESSENTIAL COMPONENTS
Data mining isn’t a simple matter. Three key components of a strong data mining efforts are:
Data. To say that data mining requires data seems obvious: But the reality of the retail industry is that the necessary data often isn’t there. Surveys, such as a recent report from The Cutter Consortium, tell us that only 15 percent of companies can say that their data warehousing projects have been a success (see Resources). For most retailers, access to a well-developed and maintained data warehouse is essential. Data warehouses are difficult to build – and even if you succeed in building them, they can be difficult to maintain as your business changes. For example, one company put a huge amount of effort into building a data warehouse that integrated clickstream data with online and offline transaction data. Then they changed to a different e-commerce platform, which in turn changed the way data was presented. It took them eight months to update their date warehouse to handle the new data representations. Meanwhile, they had no data on their customer interactions.
Algorithms. A data mining system is only as good as its algorithms – and creating the sophisticated algorithms that make up an effective data-mining system is no easy task. In particular, when dealing with customer behavioral data, which can encompass 100 dimensions or more, you need algorithms that are capable of dealing effectively with high-dimensional data. Further, it’s important that these algorithms be able to work with business constraints and rules. Simple statistics don’t work. Knowledge of the business constraints, of the relations between products, and of the various behavioral segments of customers is a must.
Skill sets. Finally, setting up and maintaining an effective data mining systems requires not only a heavy resource investment, but also several distinct skill sets. You need people who have experience in data warehousing and databases; statisticians who know how to create algorithms, deal with online analytical processing cubes, and choose the right algorithms for each task; and business users who understand how to accomplish business goals (such as increasing customer loyalty and average order size). Furthermore, each of these groups needs to be able to understand and communicate with the others.
The complexities involved in establishing excellence with each of these components reveal why some retailers are looking at outsourcing options. Outsourcing can help companies avoid some of the risks, difficulties, up-front capital and talent expenditures involved in setting up a major data-mining project. If effective, outsourcing can free the business to focus on the true sources of competitive advantage: expertise in business strategy, market analysis, and unique understanding of customer behavior. Data mining’s history has unfortunately shown that companies become too enamored with technology and spend the balance of their time and resources on making the technology work – thereby slowing progress on getting to the real business purpose for the data mining effort.
REINVENTING FAMILIARITY
The ultimate vision behind all this advanced data mining technology is quite familiar and surprisingly simple. Before the age of interactive technologies that generate lots of interaction data, the competitive advantage of a simple retailer store lay in how well the store manager knew the customers, understood their needs, and adapted the business strategy with the market. As technology allowed businesses to scale to larger numbers and many more interactions, businesses lost the simple but effective equivalent of a store manager who’s thoroughly familiar with customers and products. The ultimate goal of data mining in this context is to bring back a proxy for that simple familiarity. Data mining algorithms can uncover the value of the massive data sets many organizations collect, particularly in retail. Unfortunately, most data stores today are write-only stores, serving as little more than high-tech data tombs.
Nothing predicts behavior like behavior predicts behavior. This maxim is familiar to astute catalogers, direct marketers, and merchandisers. Unfortunately, its practitioners have failed to move beyond simple demographic-based segmentation approaches. Data mining makes it possible to leverage segments based on behavioral groups. To take full advantage of the promise of data mining, comprehensive view of your customer base – both as a whole and for the various channels and segments that is comprises. The results can make a dramatic difference in your bottom line.
2003-05-13-Optimizing-Customer-Insight
View on www.business.highbeam.com