From RFM (Recency, Frequency, Monetary) to Machine Learning

Introduction

Traditionally, the clustering of customers and the choice of the most relevant promotions are based on “RFM” parameters. Where R stands for Recency, i.e. how recent the last purchase was, F for Frequency, the number of purchases per month/year (or the average days between one and another) and M for Monetary lastly indicates the average value of each purchase. With traditional information systems, it was important to describe purchasing behavior with few significant data, in order to make it easy to write computer programs able to make the best decisions for each customer.

It is now possible to do much more by way of Machine Learning (ML), a subset of Artificial Intelligence (AI). As described in http://akite.net/en/intelli-tail-artificial-intelligence-for-retail/, it is emerging rapidly thanks to ever more affordable services provided by the Cloud.

Wide and forward vision.

By including additional data such as, for example, the detailed purchase history of each customer, the reaction to previous promotions and, where available, data such as age, sex, profession, … recommendation lists can be created (who bought product A, often also buys product B) used by the most advanced e-commerce sites. It is also very possible to define much more significant customer groups (clusters) than with simple RFM methodology, extracting significance from complex data sets (patterns) through ML techniques. In the same way, a computer can now assess gender, age and even a person’s mood by analyzing the pixels that make up the image and even recognize, while fully respecting privacy, whether the same face has already been framed a few minutes before in other areas of the store. Compared to such pattern recognition, distinguishing the different customer types from their purchase trail is much simpler. It is also clear that traditional algorithms are too complex to program and they would possibly already be old because consumers are constantly evolving.

Moreover, traditional BI (Business Intelligence) techniques look at the past and, even if the graphical representations of data are more and more effective, time and attention are required to make decisions. A modern information system must be able to make relevant proposals as if it were an experienced consultant able to see what is going to happen.

Lastly, all these calculations often have to be instantaneous, while the customer is selecting an item, or is already at the POS and there is the requirement to print a coupon that takes into account not only past history but also newly purchased products.

Data: as plentiful as possible, but significant

By analyzing the correlations of large data sets, ML creates a mathematical model able to describe the phenomenon in question in a more or less complete and precise way. This training stage or, in other words, the “ingestion, digestion and absorption” of data, requires a lot of computing resources. Once the algorithm has been created, the feeding in of new data almost instantly produces the prediction, classification, recognition and understanding of intention.

To a certain extent we are what we eat, and in the same way an algorithm is more or less efficient and effective depending on the data provided in the learning phase. A problem arises when the data are subject to biases or errors, whether conscious or unconscious. The results will only perpetuate these distortions of reality. Another type of problem is when data are not significant and do not cover the full range of aspects that influence the phenomenon or, in other words, contain little “substance and variety.” Moreover, if some data are mutually dependent on each other, the algorithm will work needlessly without absorbing information. The “Inventory turnover ratio” and the “Average days to sell the inventory”, for example, are the same entity from two different points of view (Average days to sell the inventory = 365 / Inventory turnover ratio). Providing both unnecessarily overloads the learning process. If instead “Forecasted days to sell the inventory” is used, calculated from the current sales forecast for each item, the two pieces of information will differ, as the future may be significantly different from the past and a forward vision is very critical.

Combining the customers’ wishes with stock availability and the risk of the goods’ obsolescence helps in the difficult task of achieving profit from a Retail company.

Why now and why in the Cloud, with a few warnings

The convergence of scientific progress with the computing power available from the cloud with no initial investment and at reasonable cost has triggered an explosion of AI and ML, which have remained more or less latent for 50 years. As mentioned in http://akite.net/en/intelli-tail-artificial-intelligence-for-retail/, the large volumes of data needed by ML training take time to move from on-premise to cloud and vice versa. For this reason, the best results are obtained when store management software and cognitive services both reside on the same cloud, as happens with aKite.

Although ML-as-a-Service from the Cloud greatly simplifies the use of these revolutionary techniques, the complexity of the underlying maths should not be underestimated and critical thinking not forgotten. Without a scientific method, it is easy to be victims of mirages or blunders. To name one of these, overfitting, i.e. the use of an overly complex model compared to the available data (as the effort is only on the computer) may fit the test data too well at the expense of future data. Data Science cannot, however, be separated from a deep understanding of Retail business in order to provide results that are actionable, valuable and of immediate impact. The breadth of required skills is probably the most critical aspect.

Moreover, many sales management systems excel in presenting the past, but are lacking in showing how the situation is developing and thus are not able to provide ML systems with a “rich and balanced diet”. Integration by design between sales data and ML ensures automation and maximum quality of the results.

What have been described here are only the first steps of ML in Retail. Amazon is showing the potential with the new “Amazon Go” concept. These stores without cashiers or self-scanning are based on visual recognition of every customer and the products they pick up or put back on the shelves. The future has just begun.