About MySales

MySales is a product that will help you:

• Analyze sales

• Use various regression models for product groups and items

• Break down sales by influencing factors

• Understand the reasons for changes in sales

• See the impact of prices and discounts

• Understand the impact of other factors, including competitor prices and exchange rates

• Build sales forecasts based on various models

• Improve promotional planning

• Order goods based on sales forecasts and safety stock

• Optimize deliveries through distribution centres and procurement logistics

• Optimize retail prices based on statistics and competitors' prices

• Analyze the price image using price index dynamics

In this documentation, you will find a description of MySales methods and algorithms, as well as practical instructions and a user guide.

MySales utilises several key machine learning algorithms for forecasting:

• A set of regressions and a decision tree used to build a multi-factor forecast

• A gradient descent algorithm used for multi-factor regression analysis

• Algorithms for calculating additive and multiplicative seasonality

• Neural network system for promotional forecasting

• Algorithm for assessing the similarity and comparison of promotional items

• Algorithms for optimising order quantities

It is worth noting that almost all MySales algorithms are proprietary, since the use of well-known machine learning methods and libraries does not provide stable results when dealing with a huge number (millions) of positions for forecasting that differ significantly from each other. Nevertheless, they are based on well-known algorithms.

The set of regressions is not limited to the use of simple and multiple, linear and nonlinear regressions. The gradient descent method is also used to build forecasts based on a larger number of factors than classical multiple regression allows. In addition, neural networks are used for both sales forecasting and promotion forecasting. There is also an algorithm for automatically assessing the similarity of positions in promotions, as well as other machine learning algorithms.

The key characteristic and unique feature of MySales algorithms is their predictable behaviour in various conditions and low susceptibility to overfitting, also known as the ‘fit effect’ or ‘overfit.’ This is achieved by the fact that machine learning in MySales is performed within the framework of logical rules and restrictions set by both system parameters and individual settings for the customer.

In addition, order quantity optimisation algorithms allow orders to be automatically adjusted to the required limits or restrictions in the most optimal way, taking into account forecasts, balances, and other parameters.

MySales uses both simple and multiple regression analysis to break down sales into influencing factors such as seasonality, trends, price, discounts, etc. 

However, the algorithms used in MySales are not limited to simple and multiple, linear and non-linear regressions. The gradient descent method is also used to build forecasts based on more factors than classical multiple regression allows. In addition, neural networks are used for both sales forecasting and a set of neural networks for promotional forecasting. As well as an algorithm for automatic assessment of the similarity of items in promotions, and other machine learning algorithms. The key differentiator and know-how of all MySales algorithms is their predictable behavior in different conditions and low susceptibility to overfitting, also known as the ‘over-fit effect’. This is accomplished by the fact that machine learning in MySales is performed within the framework of logical rules and restrictions set by both system parameters and individual settings for the customer.

Regression analysis is a statistical method used to assess the relationships between variables. It includes a variety of methodologies for modelling and analysing the behaviour of several variables to determine the relationship between a dependent variable and one or more independent variables. 

Regression analysis helps to understand how the value of the dependent variable (or ‘target variable’) will change when one of the independent variables changes, while the other independent variables remain fixed. In most instances, regression analysis estimates the conditional mathematical expectation of the dependent variable from the values of the independent variables; that is, the average value of the dependent variable when the values of the independent variables are fixed. 

Less frequently, the emphasis is placed on a quantile or other parameter of the conditional distribution of the dependent variable given the independent variables. In all cases, the goal of regression analysis is to restore the functional relationship between the target variable and the independent variables, or to determine the regression function.

Regression analysis is used for forecasting; the use of regression analysis significantly overlaps with the field of machine learning. Regression analysis is also used to identify and study relationships between a dependent variable and independent variables. Given limited conditions, regression analysis can be used to determine causal relationships between independent and dependent variables. Nevertheless, this can lead to errors or false connections, so caution should be exercised. As an example, correlation does not imply causation.

Regression analysis is widely used in many fields, such as aerospace and military industries, aviation and astronomy, to predict or understand the influence of external or internal factors. The effectiveness of regression analysis methods in practice depends on the data set for analysis and the regression approach chosen to apply to that data set. Since the exact form of the data generation process is usually unknown, regression analysis often relies to some extent on assumptions about that process. 

These assumptions can sometimes be tested if sufficient data is available. Regression models for forecasting are often helpful even if the assumptions are violated to some extent, although they may not be optimal.

An artificial neural network is a mathematical model, as well as its software or hardware implementation, built on the principle of the organisation and functioning of biological neural networks — networks of nerve cells in a living organism. This conception arose during the study of processes occurring in the brain and attempts to model these processes.

At MySales, neural networks are used to build multi-factor sales forecasts for items where it is important to determine the synergy effect of two or more factors, such as seasonality and discounts, as well as to forecast promotional growth for promotional items for which no comparable and similar promotional items can be found in the past.

The main strength of neural networks compared to regression analysis methods in sales forecasting is their ability to figure out the synergy effect. However, this same strength also leads to the main issues with neural networks: a higher tendency to over-fit, low transparency and poor interpretability of the results produced by the neural network.

The key distinction of MySales' patented neural network algorithms is that they allow the neural network to be trained within a set of rules, which makes the results produced by the neural network more understandable and predictable for the user.

Additionally, MySales enables the visualisation of a neural network trained to predict promotional growth, allowing users to see which factors were included as input neurons and how they influence the final result (2 output neurons), with the aim of making the neural network training results more comprehensible to the user:

Open

A model is a combination of predictors, either input or calculated variables, that influence sales and are selected to build a forecast.

MySales chooses the best model for each level: Group/Store, SKU/Store, Group/Entire Network, Group/Region, SKU/Entire Network, SKU/Region, but you can set your own models at these levels.

You are able to select your personal models for each product group, SKU, or even for each level: Group/Store and SKU/Store.

Consider that MySales already selects the best model that suits your sales. Therefore, by changing the model, you may obtain less forecast accuracy but greater sensitivity for certain predictors that you include in the model.

Including too many predictors in the model does not guarantee better forecast accuracy. A more optimal approach is to include the most important predictors in the model and exclude the rest, rather than including all predictors that have a potential impact.

You can add more models to select in MySales. The number of models is limited only by the computing power. If you have added a new model, check how many positions this model applies to. If MySales has not selected this model for any of the positions, it significa that this model does not improve the forecast.

To add a model, contact your MySales administrator, as this requires editing the “options.js” file in the root directory of the application. Always keep a backup copy of the “options.js” file when adding models

A predictor is an independent variable that is uploaded to MySales from the source system or calculated for a specific model.

Typical predictors in retail are:

• Seasonality – calculated based on historical sales

• Trend – calculated based on non-seasonal sales

• Price, discount – imported from the source system

• Price impact, discount – calculated using non-linear regressions based on price

• Price change – percentage change in price for a given week relative to the average price over the last few weeks

• Number of items – number of items in a product group

• Balance – imported from the source system

• Exchange rate – currency exchange rate from an external source

• Change in exchange rate – percentage change in currency exchange rate for a given week relative to the average exchange rate over the last few weeks

• Sales share – percentage of sales of a given product in the group

The system automatically sets a default set of predictors. These can be modified to suit the customer's needs.

The following predictors are standardized:

• Seasonality (seas) - calculated based on historical sales (additive seasonality);

• Base (avg) - base sales level (as average sales for a given period);

• Trend (inc) - calculated based on non-seasonal sales;

• Balance (stock) - loaded from the database;

• Price (price) - loaded from the database;

• Actual price (a_price) - calculated as the price including the discount;

• Discount (disc) - loaded from the database (numerical value);

• Discount in % (disc_p) - discount in percent, calculated based on price and discount;

• Goods receipts TRX (trx) - loaded from the database;

• Price change (price_chg) - percentage change in price for a given week relative to the average price for the last few weeks;

• Group sales (parent_group) - loaded from the database;

• Region sales (parent_reg) - loaded from the database;

• Number of items (cnt) - number of items in the product group at the Group level;

• Number of stores (stores) - number of stores at the region level (for regional forecasts);

• Multiplicative seasonality (seas_mult) - calculated based on historical sales;

• Availability of an item in the assortment matrix (assrt) - takes the value 0 if the item is not in the assortment matrix and 1 if it is in the assortment matrix;

• Season type (seas_type) - value 0 for normal season, -1 for low season and 1 for high season;

• Promotion share (promo_ratio) - percentage of days in a week on which a promotion was held;

• Availability (avail) - value of product availability in stores (takes the value 0 or 1);

• Exchange rate (rate) - currency exchange rate from an external source;

• Exchange rate change (rate_chg) - percentage change in the exchange rate for a given week relative to the average exchange rate for the last few weeks;

• Average price per group (grp_price) - calculated as the average price for all products in the group;

• Average discount per group (grp_disc) - calculated as the average discount for all products in the group;

• Ratio of product price to group price (grp_price_r) - calculated as the ratio of the product price to the average price per group;

• Number of items (grp_cnt) - number of items in the group for the SKU level;

• Temperature (temp) - average air temperature for a given week from an external source;

• Temperature change (temp_chg) - difference between the average temperature for a given week and the average temperature for the last 13 weeks;

• Rain (rain) - the proportion of days in the week when it rained;

• Snow (snow) - the proportion of days in the week when it snowed;

MySales verifies the predictors you include in the model for each product group and SKU, and if the impact of any predictor differs from the expected one, or if multicollinearity is detected between predictors, MySales automatically excludes these predictors from the model.

To avoid this, recalculate the forecast for the required group or SKU after you have set the set of predictors in the model and check whether MySales uses the set of predictors you have selected and whether the forecast is accurate.

In summary, consider the following rules when selecting predictors for your models:

1. Avoiding predictors that correlate with each other in the model, for example, do not include predictors such as “Discount” and “Discount %” at the same time, as they have a strong correlation

2. Avoiding including predictors in the model that have a non-linear (exponential, logarithmic, power) effect on your sales for certain product groups or SKUs; instead, include the effect of these variables, for example, if price has an exponential effect on your sales, include the “Regular price” predictor instead of the “Price” predictor.

3. Eliminate predictors that do not affect your sales or whose impact is too low; this will improve the accuracy of the model.

You are able to add more predictors, although this would require more effort than simply adding models based on combinations of existing predictors.

In order to add a predictor, you are first required to prepare the data in the source system for loading, add it to your SQL file, and configure the options file.

Occasionally, some coding may be required to convert the predictor into the appropriate data format.

The cannibalization effect is one of the most challenging analytical tasks in retail. The concept of cannibalization emerged in the 1970s and referred to the effect of reduced demand for one product as a result of adding another product to the range. This usually happens when a product is introduced that easily replaces other products. For example, when a new item of eggs that is cheaper than all the others is introduced into the range, you will see how much sales of all the old eggs in the range will fall. Cannibalization occurs when the new product is similar (or a substitute) and has the same target audience.

The cannibalization predictor is very difficult to take into account in a forecast. And it is not always possible to do so. The cannibalization forecast is based on three predictors: the average price in the group, the number of items in the group, and the ratio of the price per item to the average price of the group. The cannibalization predictor can only be used if all the predictors on which the forecast is based are highly predictable. Otherwise, the cannibalization forecast may be inaccurate.

It is also necessary to understand that the more accurate the forecast, the more prone it is to errors. Therefore, even if taking cannibalization into account in the forecast provides greater accuracy, it is sometimes better not to take it into account. The forecast will be less accurate, but the safety margin will increase.

There are two cases where it makes sense to take cannibalization into account in the forecast:

• If there are homogeneous items in product groups and there is a clear understanding of the cannibalization effect;

• If the chain uses assortment management with assortment matrices (but this can only be done with a high-quality forecast of the average price of the group and the number of items).

In any case of using cannibalization, the user must understand that it is impossible to take it fully into account and that its use will be beneficial in the event of a visible improvement in the forecast.

• Average price of the group

• Number of items in the group

• Group price coefficient - the ratio of the item price to the average price in the group

Cannibalization is taken into account in price optimization in price cross-elasticity. In the price optimization module, the user can see the relationships between specific products. Cross-elasticity is described in detail in the Cross-elasticity section.

The system can directly account for the cannibalization effect of interchangeable and similarly priced items.

To do this, it is possible to combine several items that are interchangeable and similar in price into one key element (KE). This combination is done by adding the KEY_ID field to the PRODUCTS table, where the SKU list is loaded. It is also possible to configure the key element reference book in the MySales DB Editor. If several items have the same key element (usually the SKU_ID of the earliest item), the forecast is built as follows:

Initially, the system consolidates sales and balance data for all items of the key element (KE).

A forecast is constructed for the KE, i.e. a forecast based on consolidated data

For weeks during which each KE item was included in the assortment matrix (AM), or if the item was removed from the AM before the forecast week of stock depletion, the system constructs an independent forecast

Next, the forecast for KE positions is adjusted so that the sum of the forecasts for all KE positions is equal to the total forecast for KE. Thus, the KE forecast is redistributed to the forecast for each KE position.

The weeks where the ‘Daily Adjustment’ predictor worked are not adjusted.

For instance, there were three CE positions in the AM and sales were distributed among them, and one of the positions is removed from the AM, the forecast for the remaining two will be increased. Conversely, if there were two CE positions in the AM and a third is added, the forecast for the previous two will be reduced.

Note 1: combining CE positions does not cancel the recommendation to assign an analogue. If no analogue is assigned to the new CE position, the forecast for it will be calculated as the average for the group with an adjustment for price.

Note 2: it is advisable to test the forecast for positions combined by a single CE, as such a combination may not always improve the forecast. For instance, even if the items are similar in price and consumer purpose but differ in consumer preferences and each has its own target audience, i.e. one item only partially cannibalises the other.

The price elasticity of demand is a measure of the response of the quantity demanded of a good to a change in its price.

Price elasticity of demand shows how much the demand for a particular good will change in percentage terms when its price changes by 1%.

Demand comes in two forms: elastic and inelastic.

For elastic demand, when the price of a good changes by 1%, the demand for it will change by more than 1%. For example, if the price of bananas increases by 10%, the demand for them will fall by 15%, or if the price decreases by 25%, the demand will increase by 40%. In this case, we say that the demand for this product is price elastic.

For inelastic demand, when the price of a product changes by 1%, the demand for it will change by less than 1%. For example, if all types of vegetable oil increase in price by 49% and the volume of demand decreases by only 19%, it can conclude that the demand for vegetable oil is inelastic in terms of price.

Information about the elasticity or inelasticity of demand for a product is very important for entrepreneurs.

If demand for a product is elastic, as in the case of bananas, the following pattern can be observed: if P (price) increases, Q (quantity sold) decreases by more than the price increases, or if P decreases, Q increases by more than the price decreases.

If the demand for a product is inelastic, as in the case of vegetable oil, then if P decreases, Q increases to a lesser extent than the price decreases, or if P increases, Q decreases to a lesser extent than the price increases.

Accordingly, to increase revenue from the sale of bananas, it is necessary to lower their prices, and for vegetable oil, on the contrary, to raise them.

The MySales system automatically calculates the impact of price changes on demand. It also calculates the theoretically optimal points at which sales in units, money, or margin will be maximised.

To determine the theoretically optimal prices using price elasticity, the ‘Price Optimisation’ module has been designed (detailed information about it is provided in the Price Optimization section).

New items are merchandise that has recently been added to the product range and has no sales history.

Our system automatically detects new items and generates forecasts for them. The system generates different forecasts for the following types of new items:

• ‘New items for which analogues have been entered’

• The system calculates their sales forecast based on sales data for similar products. When building a forecast, the system takes into account the influence coefficient of each of the analogues, which is set when adding a new item (adding analogues is described in detail in the New Items section).

• ‘New items for which no analogues have been created, there are no analogues and there is no sales history in the network;’

• For such positions, the system automatically finds positions that are in the assortment matrix, have less than 4 weeks of sales, and do not have an analogue specified. The system then forecasts them as the average position for the group, but taking into account the price of the new position (if it is specified for the new position; if not, it builds on the average value of the group). To do this, the system takes sales at the Group/Store level and divides them by the number of items in stock, then recalculates the group forecast taking into account the price of the new item and applies it to the new item.

• ‘New items for which there are no analogues, but there is a sales history across the network;’

• For such items, the so-called ‘Distribution Extension’ is used in the forecast. This option is used for more accurate forecasting of a new item in a store if this item has already been used in other stores in the region.

  Distribution expansion is used if the historical sales for a new item in a store are less than a defined number of weeks (the number of weeks is set using the forecast configuration file), but there are sales in the region; the system considers this item not as completely new, but as an expansion of the store's assortment.

  Talking about items that fall under distribution expansion, the forecast is calculated as a forecast for the item from the region and applied to the store using the following proportion: The Sales/Forecast value at the SKU/Region level is multiplied by the Sales/Forecast at the Group/Store level and divided by the Sales/Forecast at the Group/Region level.

Upon login, MySales downloads your historical weekly sales for each group and SKU, inventory data, and additional predictors (influencing factors, variables) that you set.

In order to break down your sales into influencing factors, MySales calculates a large number of variables:

• Additive seasonality

• Trends based on linear and nonlinear regression for non-seasonal sales

• Linear and nonlinear regression by price, discount, and other predictors that you have configured

• Linear (or non-linear) regression for calculating positions in product groups

MySales then includes these variables in various regression models, which you can view on the SETTINGS page, tests each model, and selects the model that best suits your sales.

The “Model fixing” parameter is located in section “10. Parameter processing”. By default, its value is undefined. This means that MySales will fix the selected model if the available history is greater than or equal to undefined weeks, or if it has been fixed manually. If the available history is insufficient, MySales will select the best model each time.

You can find this parameter on the SETTINGS page.

Upon testing each model, MySales builds a forecast for the past 52 weeks and calculates the Mean Absolute Percentage Error (MAPE) for the forecast for each model. Then, based on your model settings, MySales adds a penalty for each model and selects the model with the minimum error, including the penalty. The penalty is a percentage that you can use to reduce the sensitivity of a specific model.

You can see the penalty percentage for each model on the SETTINGS page. To view the models being explored and selected, use the Models list button on the MY SALES page.

You can find this parameter on the SETTINGS page, in the “6. Other Forecast Parameters” section.

This parameter can be changed to 3-6 weeks when building a short-term forecast, and increased to 9-13 weeks for a medium-term or long-term forecast.

There are several options, the simplest of which is to take the forecasted quantity for the delivery period (from the nearest delivery date to the next delivery date) and add the Mean Absolute Deviation (MAD) or Mean Absolute Percentage Error (MAPE) as a safety margin. Your target stock level is now ready. Then, simply subtract the current stock level and you will have the required quantity of goods to order. If the forecast accuracy is high, using MAPE is preferable when you need to increase the safety stock for forecasted peaks. The usage of MAD is preferable when the forecast accuracy is lower, or sales are low, or sales are expected to be smoother without high peaks.

Examples

Here is an Example 1:

• You order goods for 1 week, every Monday.

• The forecast for the next week is 10 items.

• MAPE is 20%, MAD is 1.2 items.

• Your order for the next week is 12 units (10 + 20%) minus the current balance.

Example 2:

• You order goods for 1 week, every Monday.

• Your forecast for the next week is 0.5 units.

• MAPE is 50%, MAD is 0.3 units.

• Your order for the following week is 1 unit (round(0.5 + 0.3)) minus the current balance.

Read the article to learn how MySales works with automatic product ordering.