Forecasts
Analyze
- 1 Analyze
- 1.1 Description & Overview
- 1.2 How we build forecast
- 1.3 Daily forecast adjustment
- 1.4 Calculating lost sales
- 1.5 Parameters
- 1.6 Results
- 1.7 Analysis area
- 1.8 Results display area
- 1.9 Forecast analysis components guide
- 1.9.1 Category 'Volume'
- 1.9.2 Category 'Value'
- 1.9.3 Category 'Original'
- 1.9.4 Category 'Regressed'
- 1.9.5 Category 'Economy'
- 1.10 Use cases
- 2 View
- 2.1 Overview & Use
- 2.2 Operation manual
- 2.2.1 Template creation
- 2.2.2 Template selection
- 2.2.3 Editing and deleting
- 2.2.4 Filter
- 2.3 Use cases
- 3 Master
- 3.1 Overview & Use
- 3.2 Operating manual
- 3.2.1 Working with Master
- 3.2.2 Filters
- 3.2.3 Adding forecast
- 3.2.4 Editing forecast
- 3.2.5 Deleting forecast
- 3.2.6 Master forecast import/ex[ort from files
- 3.2.7 About revisions
- 3.3 Use cases
- 4 New
- 4.1 Overview & Use
- 4.2 Operation manual
- 4.2.1 Adding new position
- 4.2.2 Editing and deleting
- 4.2.3 Filter
- 4.2.4 Load data from file
- 4.3 Use cases
- 5 Additional load
- 5.1.1 Overview
- 5.2 Operating manual
- 5.2.1 Working with page
- 5.2.2 Filters
- 5.2.3 Working with files
- 5.3 Use cases
- 6 Enabled items
- 6.1.1 Overview & Use
- 6.2 Operating manual
- 6.2.1 Working with page
- 6.2.2 Filters
- 6.3 Use cases
- 7 Safety stock adjustments
- 7.1.1 Overview & Use
- 7.2 How we apply SS Adjustments
- 7.3 Operating manual
- 7.3.1 Setting coefficients at the SKU-All stores level
- 7.3.2 Setting coefficients at the Store level
- 7.3.3 Setting coefficients at the SKU-Store level
- 7.3.4 Import coefficients from a file
- 7.3.5 Import coefficients at SKU-All stores level
- 7.3.6 Import coefficients at Store level
- 7.3.7 Import coefficients at SKU-Store level
- 7.4 Use cases
- 8 Forecast Adjustment
- 9 Anomaly & Excluded
- 10 Bad positions
- 10.1 Overview & Use
- 10.2 Filters
- 11 Statistics
Description & Overview
Page "Analyze" is designed for building forecast models and in-depth study of forecast influencing factors. The page includes two elements:
"Parameters" area is designed to select and configure the basic parameters for the forecast calculation;
"Results" area is designed to show the forecast results, factors that affecting the forecasts, as well as to study their nature.
How we build forecast
MySales has the most technologically and mathematically advanced system core (engine), tested on hundreds of millions different positions and stores, which is responsible for building a stable quality forecast for each position in each store.
The forecast is built at different levels, starting with the most aggregated levels, ending with the most detailed:
Product group - all stores
Product group - region
Product group - store
SKU - all stores
SKU - region
SKU - store
Forecasting is an automatic, multi-stage, iterative process that has a hierarchy and works with a huge data array. When building a forecast, all available history is analyzed at the high levels, up to the last 3-4 years of sales. This is a huge amount of data. Just imagine. If you have, for example 100 stores and 10,000 SKUs, then this is one million combinations multiplied by 3-4 years of sales history. At each recalculation of all forecasts, MySales analyzes gigabytes of information for a small retail chains and terabytes of information for chains with hundreds of stores.
Describing the MySales forecast calculation algorithm, we would like to warn you against trying to implement such an algorithm yourself. Even if you manage to implement such an algorithm in a reasonable amount of time, it will take years to test it on millions of different positions, catching errors, and also spend a lot of effort on optimizing calculation performance to bring it to an acceptable level before you can get economic benefits from using it.
We can distinguish the following main stages of building the MySales forecast, which system performs for each combination at each level:
Downloading and preparing data from the DBMS, as well as preparing the MySales file storage, which is used to optimize the forecast calculation speed, as well as to reduce the load on the DBMS when processing huge amounts of data. Not only historical sales, prices, discounts, balances, arrivals are loaded, but also the product hierarchy, product directory, geographic hierarchy, store directory and external data (weather, macroeconomics, competitor prices)
Inclusion of data for analogues by stores or by position. At the same time, the system always sees the latest sales data for a new store or new position and uses analogues data only for those periods where there is no information about the new product
Formation of data packages for forecast calculation in a multiprocessor environment using multiple (usually from 3 to 9) threads
Primary cleaning of sales data from key influencing factors to calculate trend and seasonality. This stage is important in order to separate the impact of prices, discounts and promotions on sales and correctly calculate seasonal uplifts. Considering that in retail the amount of historical data for many SKUs is rarely large enough to exclude periods where the influence of discounts or prices took place due to the constant rotation of the assortment. So, MySales, instead of excluding it, cleaning forecast from the influence of these factors in order not to reduce the already limited history sales, excluding only the most extreme periods. Also, at all stages, when analyzing sales, MySales excludes periods with significant sales losses due to the fact that the product was absent or it was not enough on stock to ensure sales. This approach gives the advantage that, in fact, the trend and seasonality are calculated correctly even if the sales history is limited to only one year. And missing periods may be filled from higher forecasting levels
Seasonality calculation. MySales considers both multiplicative (seasonal coefficient) and additive seasonality. This is necessary choose the most optimal method of seasonality application after analyzing whether the volatility of the seasonal periods increases along with the trend (general growth or decline in sales). Moreover, for the product groups level, seasonality of the average price of the group is also calculated, which is necessary to predict the average price of the group over a long horizon, since seasonal peaks usually have a higher average price in the group.
Calculation of the trend of overall growth or decline in sales. It is also carried out after cleaning sales from the main influencing factors (price, discount, promotional), as well as after cleaning from seasonality. The system also calculates average sales and median using cleared sales
Filling in the original matrix of historical values of predictors (influencing factors). A number of predictors are calculated values, for example, the ratio of the price for the current period to the average price for previous periods
Analysis how sales depends on influencing factors (price, discount, weather, macroeconomic factor, which most often acts as the exchange rate, cannibalization, etc.). When calculating each influencing factor, the system selectively cleans it of other, most significant influencing factors, for example, of promo and seasonality. When analyzing price dependence, the system also analyzes the trend of inflation or deflation in order to clear historical prices from such an impact. A number of factors are analyzed separately for the low, high and medium seasons, so that price elasticity, for example for ice cream, differs for winter and summer
If it is not possible to determine the dependencies of the influencing factors at detailed levels, then the system takes key ones (for example, the influence of price, seasonality) from higher levels. For example, for the SKU-store level, such dependencies can be taken from the SKU-region or SKU-all stores level, from the group-store, group-region, or group-all stores levels
Filling the target matrix of future predictor values. Here, a differentiated approach is used, which for some factors can be a simple average or median, for some - predicted values, and for some - information that was entered by the user in the customer’s system and loaded into the MySales data warehouse. It is recommended that you always load into MySales the assortment matrix, future prices as soon as they become known and enter the data about promotional activities
Calculation of correlation coefficients and formation of an automatic forecasting model based on these coefficients using those factors that affect sales
Testing automatic, as well as pre-configured forecast models. In this case, the predictive model is understood as a set of influencing factors.
Choosing the model that gives the best accuracy on past sales. When evaluating each model, periods where the product was not enough to ensure sales, as well as promotional periods that have the greatest impact on sales, are excluded
After the best model is selected, MySales builds forecast for future periods
A forecast is also carried out with a number of actions aimed at ensuring its stability and reliability, for example, calculating the minimum sales values, calculating the autocorrelation of the model error, in order to adjust the forecast for the coming weeks and make it more accurate
The next stage is the calculation of promo uplifts. At this stage, the system applies the promotion coefficients of uplift to the forecast generated by a set of neural networks (Dusya), or data from comparable promos, if such was found. Promo uplifts are adapted to the individual characteristics of each predicted position in each store to take into account the individual characteristics of different combinations and their sensitivity to promotional factors. The minimum and maximum limits of the promotion uplifts are also calculated to ensure a stable result for new positions where the history of the promotion is not enough
After calculating promo uplifts, the system performs rebalancing of the model taking into account all the influencing factors. This is necessary to balance the influence of the price and the promo effect in the promo forecast, because they often have a high correlation
At this stage, the forecast is ready, now we calculate the safety stock. SS is calculated as the standard deviation of the forecast from sales in previous periods. SS is also, as in the example above with ice cream, different in summer and winter, so the forecast is divided into 3 ranges: high, low and medium and safety stock is calculated separately for each range
Further, the forecast is used to calculate price recommendations so that you can determine the most optimal price for sales in value or margin
The final step is to calculate the possible economic effect in the past, in the form of increased sales, or sales losses for periods in which influencing factors were not known to the system, as well as a stocks reduction
It is also worth noting that the system has a separate algorithm and calculation sequence for new positions that do not have an analogue: for such positions, the system uses the sales of the average position in the product group, adjusting them to the price elasticity of the group using the specific price of the new item, and also applying a number of other restrictions and calculations to make the result more stable and accurate.
There is also a separate forecasting algorithm for expanding distribution. Example: a position was sold in 10 of 30 stores in the region, given the good dynamics and potential, the category manager decided to list it in all 30 stores in the region. In this case, the system uses sales dynamics at the level of SKU-region, group-region and group-store to build forecast for this position in a store where there is no sales history and this position is being listed for the first time.
The set of all factors that the system takes into account is set by the user and can be individually adapted. The default set of factors is described in the "Default Predictors" section.
Daily forecast adjustment
After completing all forecast calculation stages, the system adjusts it for the upcoming weeks (usually two, which is a configurable parameter) based on the most recent days of sales data. To do this, the system loads sales data at the SKU-store-day level for the last 28 days (configurable parameter) and selects the most relevant 7 (or fewer, also configurable) days of sales:
Separating promotional and non-promotional sales. Past promotional sales adjust only the promotional forecast for the next one to two weeks, while regular sales adjust the regular forecast accordingly.
Filtering out days with insufficient stock. The stock level must be greater than zero and also exceed 30% of the average daily sales (configurable parameter). In some cases, a minimum absolute stock level can be set as a parameter.
Handling cases of insufficient stock. If sales occurred despite low stock, the system may still consider them under certain conditions. For example, if a given day’s sales exceeded 75% of the forecast for that day, the system applies an upward adjustment factor. The assumption is that if stock had been sufficient, sales would have been higher:
By 50%, if the product likely ran out in the second half of the day.
By 25%, if the product likely ran out at the end of the day.
Filtering out days with significant price variations. Days where the price deviation exceeded 15% (configurable parameter) are excluded.
Calculating lost sales
For days when an item was part of the assortment matrix in the last 1–2 weeks (configurable parameter), the system calculates lost sales or missed demand for days when there was no stock or an insufficient stock level.
Similar to the Daily Forecast Adjustment calculation, insufficient stock means that either:
The stock level was zero at the end of the day, or
The stock was less than 30% (configurable parameter) of the forecasted sales for that day.
When distributing the weekly forecast into daily values, the system also applies day-of-week coefficients and considers price elasticity if the price changes during the week.
If the system finds valid sales data from the last 28 days (configurable parameter) that meet the required criteria (i.e., sufficient stock was available, promotio