Retail Demand Modeling

I've seen countless companies relying on outdated models or gut instincts for price changes. That often leads to tactical, knee-jerk pricing, missed profits, or constant battles to justify pricing & promotional plans to supply chain partners. I just recorded a quick video explaining exactly how we combine four different approaches to model elasticity accurately: 1. Double Machine Learning (DML) - Delivers a robust causal estimate by predicting sales and price from confounders, then regressing the residuals. - We typically build one DML model per SKU. In our experience, this often reflects real-world behavior best. 2. Log-Log regression models - It is simple and interpretable - perfect if you have lots of historical data, a high volume of transactions, or price variation. - The log price coefficient directly translates to elasticity. It is quick to implement, though it often oversimplifies and is not a good method for B2B. 3. ElasticNet - A regularized linear model balancing Lasso and Ridge methods. - If you have many variables, such as our promos, competitor promos, distribution, comp distribution, etc., it helps prevent overfitting. 4. Random Forest - Handles non-linearities pretty well without having to do complex data engineering. - We use price perturbation, simulating different price points to see how predicted demand changes, thus estimating implied elasticities. In the video, I also share how we compare the four methods, track metrics like RMSE or MAPE, and deliver scenario-based recommendations about price, promotions, and competitive moves, helping you go from reactive to proactive pricing. The real payoff is that you can: 1. Proactively manage pricing: estimate the impact of competitor actions and optimize your strategy. 2. Maximize promotional ROI: estimate what truly drives incremental volume vs. what's wasted spend. 3. Earn insights-backed credibility: support your pricing with robust elasticity metrics that show retailers how you got to your recommendations. I'd love to hear your thoughts. If you're ready to take a deeper look at these elasticity models (complete with a whitepaper, sample code, and practical examples), check out the comment section for links and more details!

In the last few weeks, several people commented on my earlier posts about how hard it is to forecast products with intermittent demand. It’s a common challenge across many sectors: luxury products like watches, high-end accessories, spare parts, niche or seasonal SKUs, and long-tail retail assortments. These are products that don’t sell every day, or even every week, but when they do, the timing and location really matter. The questions kept coming up, so I took some time to dig into the research and understand it better. Specifically, I wanted to answer: – What makes intermittent demand so hard to forecast? – Why do many standard models and metrics fall short? – What methods were actually designed for this kind of demand? – And when should each one be used? This article covers: – What intermittent demand looks like and how it behaves – Which models are commonly used (like Croston, SBA, TSB) and which are more advanced (like iMAPA, HES, ML, or ZI models) – A simple visual decision guide for choosing the right model – The best error metrics for zero-heavy time series (spoiler: not MAPE) – And what practitioners should actually do differently in real-world settings If you’re in luxury, spare parts, retail, or work with slow movers, this one’s for you. Curious to know, how are you managing intermittent SKUs forecasting?

The Hidden Mathematics Behind Consumer Behavior: Why Your Demand Forecasts Are Failing A deep dive into non-stationary time series and the Augmented Dickey-Fuller paradox After deep-diving into the Rossmann Store Sales dataset on Kaggle (1M+ records across 1,115 stores), I discovered something that fundamentally changed how we should approach retail demand forecasting. The problem: Traditional ARIMA models assume stationarity, but consumer behavior exhibits structural breaks that violate this core assumption. The mathematics behind the breakthrough: When we decompose our sales time series Y(t), we often see: Trend component: T(t) = α + βt + εₜ Seasonal component: S(t) following Fourier decomposition Irregular component: I(t) ~ N(0, σ²) But here's where it gets interesting... The Augmented Dickey-Fuller Test revealed something unexpected: Our null hypothesis H₀: γ = 0 in the regression: ΔSales(t) = α + βt + γSales(t-1) + δ₁ΔSales(t-1) + ... + δₚΔSales(t-p) + εₜ The counterintuitive result: 68% of Rossmann stores showed unit roots (non-stationary), but differencing wasn't the solution. The insight: Instead of forcing stationarity through differencing, we embraced Vector Error Correction Models (VECM) with cointegration relationships: ΔX(t) = αβ'X(t-1) + Γ₁ΔX(t-1) + ... + Γₖ₋₁ΔX(t-k+1) + εₜ Where β represents long-run equilibrium relationships between variables. The practical impact on Rossmann data: ✅ Identified optimal promotion timing patterns ✅ Discovered hidden seasonality in German retail cycles ✅ Predicted competition effects 4-6 weeks in advance The visualization that was interesting: Our phase space reconstruction using Takens' theorem revealed hidden attractors in retail sales patterns - cyclical behaviors invisible in traditional time domain analysis, particularly around German holidays and promotional periods. Key takeaway for fellow data scientists: Sometimes the mathematics tells us to question our assumptions rather than force our data into traditional frameworks. Non-stationarity isn't always a problem to solve - it's often information to leverage. What's your experience with retail time series forecasting? Have you worked with the Rossmann dataset or similar retail challenges? My code in google colab: https://lnkd.in/gZDHsgeR #DataScience #TimeSeries #MachineLearning #RetailForecasting #Kaggle #RossmannDataset #Statistics #Mathematics #VECM #Cointegration #RetailAnalytics #ForecastingModels #DataDriven #AdvancedAnalytics #Econometrics #BusinessIntelligence #DataStrategy #KaggleCompetition

A supply chain team at a fast-moving consumer goods (FMCG) company was struggling with frequent stockouts at retail stores, despite having enough inventory at distribution centers. The assumption was that warehouses weren’t replenishing stores fast enough, but an SQL-based analysis revealed deeper issues in demand forecasting and replenishment logic. Solving Stockout Issues with Data Analytics 1️⃣ Identifying Mismatch Between Demand & Supply We analyzed sales vs. inventory levels at retail stores. SELECT store_id, product_id, SUM(sales_quantity) AS total_sales, SUM(warehouse_shipments) AS total_shipments, (SUM(warehouse_shipments) - SUM(sales_quantity)) AS inventory_gap FROM sales_data JOIN inventory_data ON sales_data.product_id = inventory_data.product_id GROUP BY store_id, product_id ORDER BY inventory_gap ASC; 🔹 Insight: Some stores were receiving too much inventory, while others were facing stock shortages. 2️⃣ Finding Delays in Replenishment Timing We checked the average time taken for stores to receive stock from warehouses. SELECT store_id, product_id, AVG(DATEDIFF(day, order_date, delivery_date)) AS avg_replenishment_time FROM replenishment_orders GROUP BY store_id, product_id ORDER BY avg_replenishment_time DESC; 🔹 Insight: Some stores experienced delays of over 7 days, causing frequent stockouts. 3️⃣ Optimizing Replenishment Logic Based on Real Demand Instead of fixed reorder points, we recalibrated inventory restocking based on real-time demand patterns. UPDATE inventory_replenishment SET reorder_quantity = ROUND(AVG(daily_sales) * expected_lead_time, 0) FROM ( SELECT product_id, store_id, AVG(sales_quantity) AS daily_sales, AVG(lead_time) AS expected_lead_time FROM sales_data JOIN replenishment_orders ON sales_data.product_id = replenishment_orders.product_id GROUP BY product_id, store_id ) AS demand_forecast WHERE inventory_replenishment.product_id = demand_forecast.product_id; 🔹 Insight: Instead of over-relying on static thresholds, we shifted to dynamic reorder calculations. Challenges Faced Inaccurate demand forecasting led to overstocking in low-sales areas and stockouts in high-demand stores. Fixed replenishment schedules didn’t account for fluctuating demand trends. Manual interventions caused delays in order fulfillment. Business Impact ✔ 20% reduction in stockouts by implementing real-time demand-based restocking. ✔ Lower warehousing costs, as inventory was better distributed across stores. ✔ Faster replenishment cycles, ensuring high-demand stores never ran out of stock. Key Takeaway: Supply chain efficiency isn’t just about moving stock—it requires demand-driven decision-making and real-time adjustments. Have you optimized inventory using SQL? Let’s discuss!

If you’re feeding raw sales data into your forecasts, you’re making a big mistake. Retail is full of noise: stockouts, listing suspensions, influencer campaigns, promotions — all of it distorts the true demand signal. The worst thing you can do is use that raw data to feed your forecasting algorithms — you'll just forecast a new stockout or end up overstocking again. As my friend Nicolas Vandeput often says (and I agree), the best forecasting models are the ones that can consume all the context data: sales units, revenue, price history, ad spend, influencer activity, inventory levels… With that context, models can actually make sense of the past and project the future. The problem? Most brands don’t have that data organized, accessible, or reliable. So what’s the next best thing? Algorithms that automatically detect and adjust for outliers — cleaning the past before predicting the future. That’s exactly what Flieber does. The moment you connect your data, we run it through anomaly detection and correction before it ever reaches the forecasting engine. And we feed our algorithms the adjusted sales, instead of the actual sales. That step alone improves forecast accuracy by up to 40%. For planners, that’s not just a nice-to-have — it’s life-changing.

𝗪𝗵𝘆 𝗕𝗮𝘆𝗲𝘀𝗶𝗮𝗻 𝗛𝗶𝗲𝗿𝗮𝗿𝗰𝗵𝗶𝗰𝗮𝗹 𝗠𝗼𝗱𝗲𝗹𝘀 𝗔𝗿𝗲 𝗦𝗼 𝗣𝗼𝘄𝗲𝗿𝗳𝘂𝗹 A common challenge in data science is dealing with #heterogeneous data, because different regions, customer segments, or product categories may have vastly different amounts of data. Traditional approaches either 𝗺𝗼𝗱𝗲𝗹 𝗲𝗮𝗰𝗵 𝗴𝗿𝗼𝘂𝗽 𝘀𝗲𝗽𝗮𝗿𝗮𝘁𝗲𝗹𝘆, leading to noisy estimates when data is scarce, or force a 𝘀𝗶𝗻𝗴𝗹𝗲 𝗺𝗼𝗱𝗲𝗹 𝗮𝗰𝗿𝗼𝘀𝘀 𝗮𝗹𝗹 𝗴𝗿𝗼𝘂𝗽𝘀, ignoring real differences. 𝗕𝗮𝘆𝗲𝘀𝗶𝗮𝗻 𝗵𝗶𝗲𝗿𝗮𝗿𝗰𝗵𝗶𝗰𝗮𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 offer a different solution. They allow parameters to vary at 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗹𝗲𝘃𝗲𝗹𝘀 𝗼𝗳 𝗱𝗮𝘁𝗮 𝗿𝗲𝗹𝗮𝘁𝗶𝗼𝗻𝘀𝗵𝗶𝗽𝘀, letting us incorporate not just the data itself but also its underlying structure, #metadata, and the way it was collected. They capture shared #patterns while accounting for group-specific differences. This flexibility makes them ideal for data that’s nested or structured across multiple dimensions. In 𝗲𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁𝗮𝗹 𝘀𝗰𝗶𝗲𝗻𝗰𝗲, Bayesian hierarchical models are widely used because they allow scientists to measure effects at different locations, over time, or at different latitudes, all while capturing broader trends. You can read about such one example here: https://lnkd.in/d6ERwa7q In a business use case, such as 𝗿𝗲𝘁𝗮𝗶𝗹 𝗱𝗲𝗺𝗮𝗻𝗱 𝗳𝗼𝗿𝗲𝗰𝗮𝘀𝘁𝗶𝗻𝗴, Bayesian hierarchical models provide: • 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻 𝗼𝗳 𝗱𝗮𝘁𝗮 𝗮𝗰𝗿𝗼𝘀𝘀 𝗿𝗲𝗴𝗶𝗼𝗻𝘀, 𝘀𝘁𝗼𝗿𝗲𝘀, 𝗮𝗻𝗱 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 𝗰𝗮𝘁𝗲𝗴𝗼𝗿𝗶𝗲𝘀, capturing both global trends and local variations. • 𝗦𝗲𝗮𝘀𝗼𝗻𝗮𝗹𝗶𝘁𝘆 𝗺𝗼𝗱𝗲𝗹𝗶𝗻𝗴, assuming common patterns across regions but also allowing for regional differences. • 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 #sparse 𝗱𝗮𝘁𝗮, borrowing information from related datasets to improve #accuracy. You can read more about this application: https://lnkd.in/dnkcKi4b In both cases, I used #PyMC for Bayesian modeling. By allowing flexibility and borrowing strength from related data, Bayesian hierarchical models offer a robust approach to #forecasting, 𝗲𝘃𝗲𝗻 𝘄𝗶𝘁𝗵 𝗹𝗶𝗺𝗶𝘁𝗲𝗱 𝗼𝗿 𝘂𝗻𝗲𝘃𝗲𝗻 𝗱𝗮𝘁𝗮. Let me know if you've used Bayesian hierarchical models, I'd love to hear about other use cases. #BayesianInference #HierarchicalModels #DataScience #MachineLearning #Forecasting #RetailAnalytics #PyMC #EnvironmentalScience #DemandForecasting #StatisticalModeling #BusinessAnalytics #GeospatialModeling #PredictiveModeling #DataAnalysis 

AI augments retail companies in many ways, but advanced demand forecasting is among the most powerful. Here’s how you can get better at predicting the future: 1. Level up your data game. No amount of investment in expensive AI models will ever fix poor data sets. The more historical data points you have, the better: Customer demand records Logistics-related data   And even external data such as Economic indicators and Weather information. Clean it by removing irrelevant information, identifying and handling outliers, and resolving any inconsistencies so you can feed the models with relevant data for accurate analysis. 2. Select your perfect model. Consider different approaches like: Regression models Time-series analyses Neural networks Assess the computational resources available, understand each model's complexity, and decide based on the unique characteristics of your data sets. 3. Train and validate your model. Split the dataset into training and testing subsets. The model is trained using the training data and then rigorously tested against the testing dataset to validate its predictive capabilities. You should also regularly update the training dataset with new information, track the model's performance metrics, and incorporate user feedback for enhanced accuracy. 4. Deploy!  Create user-friendly interfaces to make the predictive insights accessible to decision-makers. Set up real-time prediction capabilities and alert mechanisms to communicate significant demand shifts effectively. The chart below illustrates a typical demand forecasting system (though it’s missing the testing and validation phase, which is super important, in my opinion). Ready to level up your foresight capabilities using AI? DM me, and we can make it a reality! ✉️ #management #ai #retai #ecommerce #digitaltransformation #RemoteNative Andreas Anding - Technical Leader with 20+ years of experience in large-scale digital transformation projects.

𝐀𝐭 𝐞𝐧𝐭𝐞𝐫𝐩𝐫𝐢𝐬𝐞 𝐬𝐜𝐚𝐥𝐞, 𝐬𝐦𝐚𝐥𝐥 𝐛𝐥𝐢𝐧𝐝 𝐬𝐩𝐨𝐭𝐬 𝐛𝐞𝐜𝐨𝐦𝐞 𝐛𝐢𝐥𝐥𝐢𝐨𝐧-𝐝𝐨𝐥𝐥𝐚𝐫 𝐟𝐚𝐢𝐥𝐮𝐫𝐞𝐬. For Target, January is not a slow start - It’s the launchpad for everything that follows. Now consider this scale: 100K+ SKUs. 2,000 stores. And nearly 𝟓𝟎% 𝐨𝐟 𝐨𝐮𝐭-𝐨𝐟-𝐬𝐭𝐨𝐜𝐤𝐬 not even visible to core systems. When demand, footfall and inventory are forecasted in silos, planning accuracy collapses. Industry-wide, that puts $𝟏𝟎𝟔.𝟔𝐁 𝐢𝐧 𝐚𝐧𝐧𝐮𝐚𝐥 𝐬𝐚𝐥𝐞𝐬 𝐚𝐭 𝐫𝐢𝐬𝐤. This is what changes when AI is applied end-to-end instead of point by point. 𝟓 𝐂𝐨𝐫𝐞 𝐔𝐬𝐞 𝐂𝐚𝐬𝐞𝐬 𝐓𝐚𝐫𝐠𝐞𝐭 𝐟𝐨𝐜𝐮𝐬𝐬𝐞𝐬 𝐨𝐧: ➤ Demand forecasting at SKU level ML models trained on 3+ years of history, weather, and events → 10–20% accuracy improvement vs. traditional methods ➤ Footfall prediction, not guesswork Store-level traffic forecasts tied directly to staffing and inventory → Dynamic workforce allocation and reduced wait times ➤ Real-time inventory ledger Ensemble ML processing 360K transactions per second to detect out-of-stocks as they happen → 4-8% sales lift from immediate inventory correction ➤ Trend intelligence, not lagging reports Generative AI surfaces emerging demand patterns early → Faster buying decisions and fewer markdowns ➤ Personalization at scale AI-driven recommendations and dynamic pricing across app and in-store → 4.3M daily app users and top-8 retail app adoption in the U.S. This only works because planning itself changes. 𝐓𝐡𝐞 𝐟𝐮𝐥𝐥-𝐲𝐞𝐚𝐫 𝐀𝐈 𝐩𝐥𝐚𝐧𝐧𝐢𝐧𝐠 𝐜𝐲𝐜𝐥𝐞 → Q1: Strategic targets, market analysis → Q2–Q3: Model training, forecasting, store segmentation → Q4: Deployment across 2,000 stores, inventory and workforce optimization → Ongoing: Real-time corrections, daily retraining, continuous learning 𝐖𝐡𝐲 𝐭𝐡𝐢𝐬 𝐛𝐞𝐜𝐨𝐦𝐞𝐬 𝐚 𝐜𝐨𝐦𝐩𝐞𝐭𝐢𝐭𝐢𝐯𝐞 𝐚𝐝𝐯𝐚𝐧𝐭𝐚𝐠𝐞 ✓ Real-time forecasting vs. annual/quarterly cycles ✓ Integrated system (demand → footfall → inventory → personalization) vs. siloed models ✓ Predictive out-of-stock prevention vs. reactive discovery ✓ Ensemble ML (thousands of models) vs. single-model approaches ✓ Continuous learning (daily retraining) vs. static models 𝐊𝐞𝐲 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲𝐬 𝐟𝐨𝐫 𝐥𝐞𝐚𝐝𝐞𝐫𝐬 - Retail AI wins don’t come from better dashboards. They come from architectures that see, decide, and act continuously. When planning becomes anticipatory instead of reactive, AI stops being a cost center and starts compounding value at enterprise scale. The opportunity is no longer theoretical. The question is which part of your planning stack still can’t operate in real time. Where do you see the biggest breakdown today: demand, inventory or execution? ♻️ Repost to help teams understand the different aspects of AI. 🔔 Follow Keith R. Worfolk - MBA, MCIS, CCIO, CISSP, CCISO, CCP for insights on unlocking value with AI & Enterprise Scale #AIinRetail #EnterpriseAI #AgenticAI

Forecasting is a common application of data science, and it's crucial for businesses to manage their inventory, especially those with perishable items effectively. In a recent tech blog, the data science team from Afresh shared an innovative approach to accurately predict demand, incorporating non-traditional factors such as in-store promotions. Promotions are common in grocery stores, helping customers discover and purchase discounted items. However, these promotions can significantly alter customer behavior, making traditional forecasting methods less reliable. Traditional models struggle to incorporate these factors, often leading to higher prediction errors. To address this challenge, Afresh’s data science team developed a deep learning forecasting model that integrates various features, including promotional activities tied to specific products. The model's performance was evaluated using a normalized quantile loss metric, showing an 80% reduction in loss during promotion periods. This example highlights the superior performance of this solution and showcases the power of deep learning in solving a critical issue for the grocery industry. #machinelearning #datascience #forecasting #inventory #prediction – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gj6aPBBY    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/gWRgTJ2Q 

Last post, I talked about the fact that forecasts require the right calculations in your inventory control engine. This is because in essence, we heavily rely on the correctness of the underlying demand model. This model describes the underlying lead time demand distribution and is used to calculate the distribution of the inventory level, so that we eventually can calculate performance metrics of our inventory system. These performance metrics then determine your inventory parameters (like the reorder level). There are many ways to create such a demand model. Important is that we always do it from a forecast, not just from historical demand. One way is to find a demand variance and mean directly from our forecast. Most forecasts methods generate a mean directly, but not a variance. One could then use the Mean Squared forecast Error (MSE) as an estimate. Then you have forecast models which can also generate a variance. And then you have emerging fields like probabilistic forecasting that produces a distribution directly. However, all of these methods and models have several caveats. I named three of them below. I would argue that forecast accuracy should be a concept that looks for how well we are able to model the lead time demand distribution in practice. I also argue that getting the mathematics in your inventory calculations correct is much more valuable than improving your traditional forecast accuracy metrics by 5% via a tedious implementation of ML forecasting algorithms. Like I mentioned in my previous post as well. 3 caveats in estimating your demand model in practice: -  Your forecast errors are positively correlated. Scaling the probability distribution over the lead time using the square root law, the way commonly done in practice, is therefore incorrect. -  When you are dealing with stochastic lead times, your lead time demand distribution should deal with this accordingly. You want the convolution of both the demand and the lead time distribution. -  The demand model is bounded by the inventory that is available. Sales is not the same as the demand and this actually asks for estimating the so-called truncated probability distribution of the lead time demand. ------------------------------------------------------------------------------------- Do you want to stay updated with insights, concepts, tips, articles or new papers on inventory optimization? Feel free to click the follow button or connect! Also, feel free to subscribe to my newsletter “The Newsvendor Problem” if you are passionate about inventory optimization and forecasting: https://lnkd.in/eAHmRFgV #Forecasting #inventorymanagement