--- type: "new-runtime-knowledge-guide" stable_id: "knowledge_guide:forecasting-causal-bandits" version: 1 generated_at: "2026-07-23" record_date: "2026-07-23" date_kind: "generated_at" slug: "forecasting-causal-bandits" title: "Forecasting, Causal Inference & Bandits" description: "The next layer after classic ML. Not just “predict the label”, but understand three different questions: what will happen next, what will happen if we intervene, and what is better to show right now with incomplete information. These are three different formulations of the problem, with different metrics, leakage risks and different language of communication with researchers and product teams." retrieval_nugget: "The next layer after classic ML. Not just “predict the label”, but understand three different questions: what will happen next, what will happen if we intervene, and what is better to show right now with incomplete information. These are three different formulations of the problem, with different metrics, leakage risks and different language of communication with researchers and product teams." track: "ml-and-decision-systems" volume: 17 --- # Forecasting, Causal Inference & Bandits ## Retrieval answer The next layer after classic ML. Not just “predict the label”, but understand three different questions: what will happen next, what will happen if we intervene, and what is better to show right now with incomplete information. These are three different formulations of the problem, with different metrics, leakage risks and different language of communication with researchers and product teams. ## 01. Correct Statement of the Problem ### 01.01 Three different questions that are often confused A lot of the confusion in applied ML starts not with the model, but with the wrong question. `Forecasting` asks: what is most likely to happen next? `Causal inference` asks: what will change if we intervene? `Bandit/policy` asks: what is better to choose right now, while we are still learning? #### Diagram ```text THREE CLASSES OF QUESTIONS Forecasting "What will be the demand tomorrow?" “how much money will the client bring?” Causal inference “Will a discount increase conversion?” "what will happen if you change onboarding?" Bandit/policy “Which offer should I show this user now?” "Which ranking option should I use in the next impression?" Similar at the product level, but mathematically these are different problems ``` ### 01.02 Why does regular iid logic break down? Classic supervised ML often assumes that the examples are independent and the distribution does not drift too quickly. In a time series, neighboring points depend on each other in time. In causal, we are not just interested in the observed correlation, but in the effect of the intervention. In bandits, the data collection policy itself affects future data. #### what exactly breaks ```sql time series: x_t depends on x_(t-1), x_(t-2), ... causal: causal observed outcome != effect of action bandits: you select action and with this you change what you see next ``` ### 01.03 Prediction, intervention, policy - three different types of output Sometimes the model needs to predict a future value. Sometimes estimate the effect size of an action. Sometimes choose the action with the best expected reward. If you mix these three outputs, you can get a technically neat, but productually useless system. | Mode | What do we give out? | What we optimize | | --- | --- | --- | | Forecast | y_hat(t+h) | forecast error | | Causal | treatment effect | effect estimate bias | | Bandit | action / policy | regret / cumulative reward | ### 01.04 Mini-dictionary: horizon, counterfactual, reward, regret To speak the same language with researchers and analysts, it is enough to remember a few central terms. They constantly pop up in articles, discussions of experiments, and product reviews. #### terms ```text horizon: how many steps forward do we predict? counterfactual: what would happen if the action was different reward: Useful framing of the chosen action regret: how much was lost relative to the best action policy: rule for choosing an action by context ``` ## 02. Forecasting: Time Series Basics ### 02.05 Anatomy of a time series: trend, seasonality, noise, shocks A time series is rarely just a list of numbers. It is typically a mixture of slow baseline changes, repeating seasonal cycles, random noise, and occasional external shocks. A conversation about forecasting often begins with the decomposition of these components. #### Diagram ```text WHAT IS THE SERIES CONSISTED OF? y_t = trend long rise/decline + seasonality day of week, month, hour + cycles longer oscillations + noise random noise + shocks holidays, promotions, accidents, releases A good model either explicitly or implicitly separates these effects ``` ### 02.06 Lags and autocorrelation: the past really helps If today's value is similar to yesterday's or to the value a week ago, then there is auto-dependence in the series. This is why lags and rolling statistics are so important: they give the model direct access to the past state of the system. #### typical features ```text lag_1 = y_(t-1) lag_7 = y_(t-7) lag_24 = y_(t-24) roll_mean_7 = mean(y_(t-6) ... y_t) roll_std_30 = std(last_30_points) calendar: day_of_week, month, holiday_flag ``` ### 02.07 One-step vs multi-step forecasting Forecasting one step ahead and forecasting 30 steps ahead are not the same task. The further the horizon, the higher the uncertainty and the more errors accumulate. In multi-step systems, you need to decide in advance whether you will build one general multi-horizon forecast or repeat a single-step forecast along the chain. #### Diagram ```sql TWO MODES one-step: y_(t+1) from past to t recursive multi-step: first predict y_(t+1) then use it to predict y_(t+2) and so on direct multi-step: separate model/head for each horizon recursive is simpler, direct is often more stable on the far horizon ``` ### 02.08 Temporary split and rolling backtest: you can’t interfere with the future in the train The usual random split for time series is almost always wrong. The future should not be included in the training of the model that supposedly predicts this future. Therefore, they use time-based split, rolling origin evaluation and `TimeSeriesSplit`-like schemes, where train is always strictly before validation. #### Diagram ```text CORRECT SERIES VALIDATION Fold 1: train = [1 ... 100] test = [101 ... 120] Fold 2: train = [1 ... 120] test = [121 ... 140] Fold 3: train = [1 ... 140] test = [141 ... 160] train grows, test always lies in the future ``` #### antipatterns - shuffle split for row - feature from the future window - walk-forward validation ### 02.09 Leakage in time series is almost always subtler than it seems The most common failure of a forecasting project is not the “wrong model”, but an unnoticeable leak of the future. Any rolling mean, target encoding, cumulative metric or exogenous signal should be considered as if you are actually living at time `t` and do not yet know the future. #### typical leaks ```sql bad: average demand of the month, if the month has not yet ended normalizer, fit on the entire row signs from post-event tables good: only information available at t feature engineering inside each fold ``` ## 03. Forecasting: Tools & Metrics ### 03.10 Naive baseline is almost mandatory In forecasting it is very easy to build a complex model that is worse than the simplest baseline. Therefore, they almost always consider at least `last value`, `seasonal naive` or `drift`. If you don't beat these primitives, the project has not yet reached the real ML win. #### simple basic forecasts ```text naive: y_hat_(t+1) = y_t seasonal naive: y_hat_(t+h) = y_(t+h-m) drift: continue the last linear trend basic question: are we really better than this? ``` ### 03.11 Two large families of forecasting models In practice, forecasting often follows two paths. First: specialized time-series models like ETS / ARIMA, which explicitly describe the dynamics of the series. Second: feature-based ML, where you build lags, rolling features and exogenous features, and then train gradient boosting or another supervised model. | Family | Strength | When it's convenient | | --- | --- | --- | | ETS / ARIMA | the structure of the series is clearly laid down | single rows, transparent statistics | | Feature-based ML | easy to add external features | many entities, complex business context | | Neural forecasting | shared patterns across many series | scale, large panels, complex dependencies | ### 03.12 ML approach to forecasting: a series turns into a supervised table Very often forecasting is solved not by a separate “magic” model, but by ordinary supervised ML. You create lags, calendar features, prices, stocks, balances, weather data and turn each time point into a table row. Then you can learn boosting, a linear model or a neural network. #### table forecasting scheme ```text row at time t: lag_1 lag_7 roll_mean_14 day_of_week holiday price promo_flag stock ... target: y_(t+h) ``` ### 03.13 MAE, RMSE, MAPE: metric changes conclusions Different forecasting metrics penalize the model in different ways. RMSE hits harder for large errors, MAE is more robust, MAPE breaks down around zero, and sMAPE partially corrects this defect. Therefore, one cannot say “the model is better” without specifying by what metric and on what horizon. #### short intuition - MAE: mean absolute error - RMSE: Big misses hurt more - MAPE behaves poorly at small values - see horizon metrics separately ## 04. Causal Inference: What Happens If We Intervene ### 04.14 Prediction ≠ Causation The usual predictive model answers the question “who buys more often?” well, but it still does not answer the question “what will happen if we send a promotional code?”. A model can find strong correlations and yet not know the true effect of an action at all. Causal inference is needed exactly where we are interested in the result of the intervention. #### Diagram ```text KEY DIFFERENCE Prediction: P(Y | X) "What kind of users usually buy?" Causal: effect of do(T=1) vs do(T=0) "What will change if we actually turn on treatment?" High predictive accuracy does not guarantee correct causal assessment ``` ### 04.15 Potential Outcomes: Every object has two worlds The basic causal intuition is simple: the same person or object has at least two potential outcomes - with treatment and without treatment. The problem is that in reality we only see one of them. The second world remains a counterfactual, and that is why causal inference is more difficult than ordinary prediction. #### designations ```text Y(1): outcome if treatment was Y(0): outcome if there was no treatment individual effect: τ = Y(1) - Y(0) ATE: E[Y(1) - Y(0)] ``` ### 04.16 Confounding: a hidden reason ruins a naive comparison If treatment is not obtained by chance, but because the object already differs in important ways, a simple comparison of the treated and untreated groups will be biased. This is confounding. Usually, causal work is largely a fight against it. #### Diagram ```text EXAMPLE CONFOUNDING observation: people with expensive tariffs buy more naive output: expensive tariff causes purchases reality: perhaps they switch to an expensive tariff more often already active and solvent clients hidden reason: user_intent/affluence/engagement ``` ### 04.17 A/B test: the gold standard of causal evaluation Randomization makes the treatment independent of hidden factors on the sample average. This is why A/B tests are so valuable: they eliminate much of the confounding without complex math. If a good experiment can be done, this is usually the best way to estimate a causal effect. #### meaning of randomization ```text random assign: T ~ Bernoulli(p) → treatment does not depend on user traits → groups are comparable on average → difference in outcomes is closer to causal effect ``` ### 04.18 When A/B is impossible: basic observational approaches If an experiment is not possible or too expensive, causal inference attempts to approach it through research design. There is no one universal method: matching, regression adjustment, inverse propensity weighting, difference-in-differences, regression discontinuity and instrumental variables solve different types of bias. | Approach | Intuition | When appropriate | | --- | --- | --- | | Matching / propensity | compare similar treated and untreated | observational data rich in covariates | | DiD | compare before/after changes between groups | policy rollout, natural experiments | | RDD | compare around destination threshold | strict business thresholds | | IV | use an external source of variation | strong confounding, but there is a valid instrument | ### 04.19 ATE, CATE, uplift: effect may vary for different people The average effect is useful, but heterogeneity is often more important in products. One segment benefits from treatment, another is neutral, and the third even gets worse. Hence the language of `CATE`, `uplift modeling`, `treatment heterogeneity` and personalized interventions. #### effect levels ```sql ATE: average effect over the entire population CATE: effect for subgroup/context uplift: increase in outcome precisely because of treatment In personalization, one is almost always interested not just in P(Y), but in ΔY from action ``` ### 04.20 Frequent causal failures Even neat teams regularly make mistakes in causal statements. Most often, this is either an incorrect assumption about parallel trends / ignorability, or post-treatment leakage, or an overly optimistic transfer of the effect from one environment to another. #### antipatterns - control a variable that treatment has already changed - ignore group selection - transfer the effect without re-validation - first write out identification assumptions ## 05. Bandits: Learning While Making Decisions ### 05.21 Multi-Armed Bandit: exploration vs exploitation The Bandit problem occurs when you need to choose an action right now, but you are not yet sure which action is best. If you always choose the current leader, you may miss the best option. If you experiment too much, you lose the reward today. It is this compromise that is called exploration vs exploitation. #### Diagram ```text CLASSICAL METAPHOR you have several “hands” of the machine: arm A arm B arm C each gives reward with unknown distribution At each step you need to decide: play the best known hand? or else check the less studied one? bandit = limited risk online learning ``` ### 05.22 Regret: the main currency of bandit systems In bandits, it is important not only how much reward you collected, but also how much you lost relative to the ideal oracle, which always knows the best action. This difference is regret. A good bandit policy makes regret grow slowly as experience accumulates. #### intuition regret ```text oracle reward: always chooses the best arm policy reward: sometimes makes mistakes while learning regret = oracle cumulative reward - policy cumulative reward ``` ### 05.23 Epsilon-Greedy: The Easiest Way to Explore Epsilon-greedy works crudely, but is very clear. With probability `1 - ε` you choose the best known arm, and with probability `ε` you randomly try something else. This is a simple baseline that is often used as a starting point before smarter strategies. #### algorithm ```python if rand() < ε: action = random arm else: action = argmax estimated_reward update estimate after observing reward ``` ### 05.24 UCB: underexplored options get uncertainty bonus Upper Confidence Bound chooses not just the arm with the best average reward, but the arm with the best “optimistic score”. If an option has been little explored, an exploration bonus is added to it. Therefore, UCB automatically balances learning and use without random rolls like epsilon-greedy. #### UCB intuition ```python score(arm) = empirical_mean(arm) + uncertainty_bonus(arm) if arm is poorly studied: bonus big if arm is studied well: bonus small ``` ### 05.25 Thompson Sampling: Explore through Probabilistic Faith Thompson Sampling stores the uncertainty distribution over the quality of each arm and samples from this belief at each step. An Arm with a potentially high reward and high uncertainty has a natural chance of being selected. This is one of the most practical bandit approaches. #### simplified diagram ```sql for each arm: sample θ_arm from posterior action = argmax sampled_θ_arm observe reward update posterior ``` ### 05.26 Contextual Bandits: the best action depends on the context In many products there is no one best arm for everyone. It is better to show different options to different users. Then bandit receives the user's context as input and learns a policy of the form `π(action | context)`. This is closer to personalization, advertising, recommendations and adaptive ranking. #### Diagram ```sql FROM MULTI-ARMED TO CONTEXTUAL simple bandit: choose best arm overall contextual bandit: user features x ↓ choose arm, the best one for x example: new user → onboarding A power user → onboarding B budget user→ offer C ``` #### typical domains - ads - recommendations - homepage modules - we need neat action-probabilities logging logic ## 06. A/B, Bandits AND Online Evaluation ### 06.27 A/B testing vs bandits: not competitors, but different tools An A/B test is good when you need to reliably measure the effect of fixed options. Bandits are good when you need to adaptively collect rewards along the way, without showing weak options for too long. They are often combined in products: first A/B for pure measurement, then bandit for adaptive serving. | Approach | Main goal | Strength | | --- | --- | --- | | A/B | measure the causal effect | pure interpretation | | Bandit | maximize reward while learning | less losses on weak options | | Hybrid | measure + adapt | realistic food compromise | ### 06.28 Offline evaluation is not equal to online success Forecasting can be fairly fairly validated by backtest. Causal - through experimental design and assumptions. Bandits and policy systems often run into the fact that offline logs are collected by the old policy, and the new policy will collect different data. Therefore, online validation is especially important here. #### main idea - offline is useful for weeding out weak ideas - online determines the real product value - do not confuse replay on old logs with true future behavior ### 06.29 Delayed feedback and non-stationarity In many real-world systems, the reward does not come immediately, but after hours or days. Plus, the environment is changing: season, competitors, interface, economic situation, new user segments. This complicates forecasting, bandits, and post-launch causal assessment. #### what breaks the system ```text delayed feedback: action now reward tomorrow non-stationarity: P(data) changes over time policy feedback loop: the system itself changes user behavior ``` ## 07. How to Choose the Right Frame ### 07.30 Decision Tree: forecasting, causal inference or bandit? When a product problem is formulated vaguely, it is useful to quickly run it through one decision tree. Usually this is enough to avoid launching an erroneous project under the guise of “let’s just make an ML model.” #### Diagram ```sql QUICK SELECTION OF POSITION Question: you need to understand what will happen next, if we don't change anything? → Forecasting Need to understand the effect of a change? → Causal inference You need to select action online, Are we still learning? → Bandit / political learning Do you just need to recognize the class of an object? → regular supervised ML ``` ### 07.31 Where is the place for reinforcement learning? Bandits are often called “RL lite”. The difference is that bandit usually does not model long sequences of states and long-term consequences through state transitions. RL is needed where the action now affects what state you will be in next, and this is repeated along the chain. #### comparison ```text bandit: context → action → immediate reward RL: state → action → next state → ... → long-term return If there are no long state dynamics, bandit is often simpler and more practical ``` ### 07.32 The main idea of the volume There is no one universal “ML model for business”. There are different classes of questions. If you need a forecast, build forecasting and think about the horizon and backtesting. If you want an intervention effect, think causal and assumptions. If you need an online selection of actions, go to bandits and regret. Correct setup saves months of useless optimization. #### short reminder - predict future → forecasting - estimate impact → causal - choose adaptively → bandit - do not confuse these tasks under the word “ML”