{ "type": "new-runtime-knowledge-guide", "version": 1, "generated_at": "2026-07-23", "volume": 17, "slug": "forecasting-causal-bandits", "title": "Forecasting, Causal Inference & Bandits", "description": "The next layer after classic ML. Not just “predict the label”, but understand three different questions: what will happen next, what will happen if we intervene, and what is better to show right now with incomplete information. These are three different formulations of the problem, with different metrics, leakage risks and different language of communication with researchers and product teams.", "track": { "slug": "ml-and-decision-systems", "title": "ML & Decision Systems", "route": "https://newruntime.com/learn/tracks/ml-and-decision-systems/", "position": 4 }, "counts": { "sections": 7, "cards": 32 }, "routes": { "html": "https://newruntime.com/learn/forecasting-causal-bandits/", "json": "https://newruntime.com/learn/forecasting-causal-bandits.json" }, "sections": [ { "number": "01", "title": "Correct Statement of the Problem", "intro": "", "cards": [ { "number": "01", "title": "Three different questions that are often confused", "tag": "Framing", "description": "A lot of the confusion in applied ML starts not with the model, but with the wrong question. `Forecasting` asks: what is most likely to happen next? `Causal inference` asks: what will change if we intervene? `Bandit/policy` asks: what is better to choose right now, while we are still learning?", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "THREE CLASSES OF QUESTIONS\n\nForecasting\n \"What will be the demand tomorrow?\"\n “how much money will the client bring?”\n\nCausal inference\n “Will a discount increase conversion?”\n \"what will happen if you change onboarding?\"\n\nBandit/policy\n “Which offer should I show this user now?”\n \"Which ranking option should I use in the next impression?\"\n\nSimilar at the product level, but mathematically these are different problems" } ] }, { "number": "02", "title": "Why does regular iid logic break down?", "tag": "Framing", "description": "Classic supervised ML often assumes that the examples are independent and the distribution does not drift too quickly. In a time series, neighboring points depend on each other in time. In causal, we are not just interested in the observed correlation, but in the effect of the intervention. In bandits, the data collection policy itself affects future data.", "blocks": [ { "kind": "code", "label": "what exactly breaks", "language": "sql", "value": "time series:\n x_t depends on x_(t-1), x_(t-2), ...\n\ncausal: causal\n observed outcome != effect of action\n\nbandits:\n you select action\n and with this you change what you see next" } ] }, { "number": "03", "title": "Prediction, intervention, policy - three different types of output", "tag": "Framing", "description": "Sometimes the model needs to predict a future value. Sometimes estimate the effect size of an action. Sometimes choose the action with the best expected reward. If you mix these three outputs, you can get a technically neat, but productually useless system.", "blocks": [ { "kind": "table", "headers": [ "Mode", "What do we give out?", "What we optimize" ], "rows": [ [ "Forecast", "y_hat(t+h)", "forecast error" ], [ "Causal", "treatment effect", "effect estimate bias" ], [ "Bandit", "action / policy", "regret / cumulative reward" ] ] } ] }, { "number": "04", "title": "Mini-dictionary: horizon, counterfactual, reward, regret", "tag": "Framing", "description": "To speak the same language with researchers and analysts, it is enough to remember a few central terms. They constantly pop up in articles, discussions of experiments, and product reviews.", "blocks": [ { "kind": "code", "label": "terms", "language": "text", "value": "horizon:\n how many steps forward do we predict?\n\ncounterfactual:\n what would happen if the action was different\n\nreward:\n Useful framing of the chosen action\n\nregret:\n how much was lost relative to the best action\n\npolicy:\n rule for choosing an action by context" } ] } ] }, { "number": "02", "title": "Forecasting: Time Series Basics", "intro": "", "cards": [ { "number": "05", "title": "Anatomy of a time series: trend, seasonality, noise, shocks", "tag": "Forecasting", "description": "A time series is rarely just a list of numbers. It is typically a mixture of slow baseline changes, repeating seasonal cycles, random noise, and occasional external shocks. A conversation about forecasting often begins with the decomposition of these components.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "WHAT IS THE SERIES CONSISTED OF?\n\ny_t =\n trend long rise/decline\n + seasonality day of week, month, hour\n + cycles longer oscillations\n + noise random noise\n + shocks holidays, promotions, accidents, releases\n\nA good model either explicitly or implicitly separates these effects" } ] }, { "number": "06", "title": "Lags and autocorrelation: the past really helps", "tag": "Forecasting", "description": "If today's value is similar to yesterday's or to the value a week ago, then there is auto-dependence in the series. This is why lags and rolling statistics are so important: they give the model direct access to the past state of the system.", "blocks": [ { "kind": "code", "label": "typical features", "language": "text", "value": "lag_1 = y_(t-1)\nlag_7 = y_(t-7)\nlag_24 = y_(t-24)\n\nroll_mean_7 = mean(y_(t-6) ... y_t)\nroll_std_30 = std(last_30_points)\n\ncalendar:\n day_of_week, month, holiday_flag" } ] }, { "number": "07", "title": "One-step vs multi-step forecasting", "tag": "Forecasting", "description": "Forecasting one step ahead and forecasting 30 steps ahead are not the same task. The further the horizon, the higher the uncertainty and the more errors accumulate. In multi-step systems, you need to decide in advance whether you will build one general multi-horizon forecast or repeat a single-step forecast along the chain.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "sql", "value": "TWO MODES\n\none-step:\n y_(t+1) from past to t\n\nrecursive multi-step:\n first predict y_(t+1)\n then use it to predict y_(t+2)\n and so on\n\ndirect multi-step:\n separate model/head for each horizon\n\nrecursive is simpler, direct is often more stable on the far horizon" } ] }, { "number": "08", "title": "Temporary split and rolling backtest: you can’t interfere with the future in the train", "tag": "Evaluation", "description": "The usual random split for time series is almost always wrong. The future should not be included in the training of the model that supposedly predicts this future. Therefore, they use time-based split, rolling origin evaluation and `TimeSeriesSplit`-like schemes, where train is always strictly before validation.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "CORRECT SERIES VALIDATION\n\nFold 1:\n train = [1 ... 100]\n test = [101 ... 120]\n\nFold 2:\n train = [1 ... 120]\n test = [121 ... 140]\n\nFold 3:\n train = [1 ... 140]\n test = [141 ... 160]\n\ntrain grows, test always lies in the future" }, { "kind": "list", "label": "antipatterns", "items": [ "shuffle split for row", "feature from the future window", "walk-forward validation" ] } ] }, { "number": "09", "title": "Leakage in time series is almost always subtler than it seems", "tag": "Workflow", "description": "The most common failure of a forecasting project is not the “wrong model”, but an unnoticeable leak of the future. Any rolling mean, target encoding, cumulative metric or exogenous signal should be considered as if you are actually living at time `t` and do not yet know the future.", "blocks": [ { "kind": "code", "label": "typical leaks", "language": "sql", "value": "bad:\n average demand of the month, if the month has not yet ended\n normalizer, fit on the entire row\n signs from post-event tables\n\ngood:\n only information available at t\n feature engineering inside each fold" } ] } ] }, { "number": "03", "title": "Forecasting: Tools & Metrics", "intro": "", "cards": [ { "number": "10", "title": "Naive baseline is almost mandatory", "tag": "Forecasting", "description": "In forecasting it is very easy to build a complex model that is worse than the simplest baseline. Therefore, they almost always consider at least `last value`, `seasonal naive` or `drift`. If you don't beat these primitives, the project has not yet reached the real ML win.", "blocks": [ { "kind": "code", "label": "simple basic forecasts", "language": "text", "value": "naive:\n y_hat_(t+1) = y_t\n\nseasonal naive:\n y_hat_(t+h) = y_(t+h-m)\n\ndrift:\n continue the last linear trend\n\nbasic question: are we really better than this?" } ] }, { "number": "11", "title": "Two large families of forecasting models", "tag": "Forecasting", "description": "In practice, forecasting often follows two paths. First: specialized time-series models like ETS / ARIMA, which explicitly describe the dynamics of the series. Second: feature-based ML, where you build lags, rolling features and exogenous features, and then train gradient boosting or another supervised model.", "blocks": [ { "kind": "table", "headers": [ "Family", "Strength", "When it's convenient" ], "rows": [ [ "ETS / ARIMA", "the structure of the series is clearly laid down", "single rows, transparent statistics" ], [ "Feature-based ML", "easy to add external features", "many entities, complex business context" ], [ "Neural forecasting", "shared patterns across many series", "scale, large panels, complex dependencies" ] ] } ] }, { "number": "12", "title": "ML approach to forecasting: a series turns into a supervised table", "tag": "Forecasting", "description": "Very often forecasting is solved not by a separate “magic” model, but by ordinary supervised ML. You create lags, calendar features, prices, stocks, balances, weather data and turn each time point into a table row. Then you can learn boosting, a linear model or a neural network.", "blocks": [ { "kind": "code", "label": "table forecasting scheme", "language": "text", "value": "row at time t:\n lag_1\n lag_7\n roll_mean_14\n day_of_week\n holiday\n price\n promo_flag\n stock\n ...\ntarget:\n y_(t+h)" } ] }, { "number": "13", "title": "MAE, RMSE, MAPE: metric changes conclusions", "tag": "Evaluation", "description": "Different forecasting metrics penalize the model in different ways. RMSE hits harder for large errors, MAE is more robust, MAPE breaks down around zero, and sMAPE partially corrects this defect. Therefore, one cannot say “the model is better” without specifying by what metric and on what horizon.", "blocks": [ { "kind": "list", "label": "short intuition", "items": [ "MAE: mean absolute error", "RMSE: Big misses hurt more", "MAPE behaves poorly at small values", "see horizon metrics separately" ] } ] } ] }, { "number": "04", "title": "Causal Inference: What Happens If We Intervene", "intro": "", "cards": [ { "number": "14", "title": "Prediction ≠ Causation", "tag": "Causal", "description": "The usual predictive model answers the question “who buys more often?” well, but it still does not answer the question “what will happen if we send a promotional code?”. A model can find strong correlations and yet not know the true effect of an action at all. Causal inference is needed exactly where we are interested in the result of the intervention.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "KEY DIFFERENCE\n\nPrediction:\n P(Y | X)\n \"What kind of users usually buy?\"\n\nCausal:\n effect of do(T=1) vs do(T=0)\n \"What will change if we actually turn on treatment?\"\n\nHigh predictive accuracy does not guarantee correct causal assessment" } ] }, { "number": "15", "title": "Potential Outcomes: Every object has two worlds", "tag": "Causal", "description": "The basic causal intuition is simple: the same person or object has at least two potential outcomes - with treatment and without treatment. The problem is that in reality we only see one of them. The second world remains a counterfactual, and that is why causal inference is more difficult than ordinary prediction.", "blocks": [ { "kind": "code", "label": "designations", "language": "text", "value": "Y(1):\n outcome if treatment was\n\nY(0):\n outcome if there was no treatment\n\nindividual effect:\n τ = Y(1) - Y(0)\n\nATE:\n E[Y(1) - Y(0)]" } ] }, { "number": "16", "title": "Confounding: a hidden reason ruins a naive comparison", "tag": "Causal", "description": "If treatment is not obtained by chance, but because the object already differs in important ways, a simple comparison of the treated and untreated groups will be biased. This is confounding. Usually, causal work is largely a fight against it.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "EXAMPLE CONFOUNDING\n\nobservation:\n people with expensive tariffs buy more\n\nnaive output:\n expensive tariff causes purchases\n\nreality:\n perhaps they switch to an expensive tariff more often\n already active and solvent clients\n\nhidden reason:\n user_intent/affluence/engagement" } ] }, { "number": "17", "title": "A/B test: the gold standard of causal evaluation", "tag": "Causal", "description": "Randomization makes the treatment independent of hidden factors on the sample average. This is why A/B tests are so valuable: they eliminate much of the confounding without complex math. If a good experiment can be done, this is usually the best way to estimate a causal effect.", "blocks": [ { "kind": "code", "label": "meaning of randomization", "language": "text", "value": "random assign:\n T ~ Bernoulli(p)\n\n→ treatment does not depend on user traits\n→ groups are comparable on average\n→ difference in outcomes is closer to causal effect" } ] }, { "number": "18", "title": "When A/B is impossible: basic observational approaches", "tag": "Causal", "description": "If an experiment is not possible or too expensive, causal inference attempts to approach it through research design. There is no one universal method: matching, regression adjustment, inverse propensity weighting, difference-in-differences, regression discontinuity and instrumental variables solve different types of bias.", "blocks": [ { "kind": "table", "headers": [ "Approach", "Intuition", "When appropriate" ], "rows": [ [ "Matching / propensity", "compare similar treated and untreated", "observational data rich in covariates" ], [ "DiD", "compare before/after changes between groups", "policy rollout, natural experiments" ], [ "RDD", "compare around destination threshold", "strict business thresholds" ], [ "IV", "use an external source of variation", "strong confounding, but there is a valid instrument" ] ] } ] }, { "number": "19", "title": "ATE, CATE, uplift: effect may vary for different people", "tag": "Policy", "description": "The average effect is useful, but heterogeneity is often more important in products. One segment benefits from treatment, another is neutral, and the third even gets worse. Hence the language of `CATE`, `uplift modeling`, `treatment heterogeneity` and personalized interventions.", "blocks": [ { "kind": "code", "label": "effect levels", "language": "sql", "value": "ATE:\n average effect over the entire population\n\nCATE:\n effect for subgroup/context\n\nuplift:\n increase in outcome precisely because of treatment\n\nIn personalization, one is almost always interested not just in P(Y), but in ΔY from action" } ] }, { "number": "20", "title": "Frequent causal failures", "tag": "Causal", "description": "Even neat teams regularly make mistakes in causal statements. Most often, this is either an incorrect assumption about parallel trends / ignorability, or post-treatment leakage, or an overly optimistic transfer of the effect from one environment to another.", "blocks": [ { "kind": "list", "label": "antipatterns", "items": [ "control a variable that treatment has already changed", "ignore group selection", "transfer the effect without re-validation", "first write out identification assumptions" ] } ] } ] }, { "number": "05", "title": "Bandits: Learning While Making Decisions", "intro": "", "cards": [ { "number": "21", "title": "Multi-Armed Bandit: exploration vs exploitation", "tag": "Bandit", "description": "The Bandit problem occurs when you need to choose an action right now, but you are not yet sure which action is best. If you always choose the current leader, you may miss the best option. If you experiment too much, you lose the reward today. It is this compromise that is called exploration vs exploitation.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "text", "value": "CLASSICAL METAPHOR\n\nyou have several “hands” of the machine:\n arm A\n arm B\n arm C\n\neach gives reward with unknown distribution\n\nAt each step you need to decide:\n play the best known hand?\n or else check the less studied one?\n\nbandit = limited risk online learning" } ] }, { "number": "22", "title": "Regret: the main currency of bandit systems", "tag": "Bandit", "description": "In bandits, it is important not only how much reward you collected, but also how much you lost relative to the ideal oracle, which always knows the best action. This difference is regret. A good bandit policy makes regret grow slowly as experience accumulates.", "blocks": [ { "kind": "code", "label": "intuition regret", "language": "text", "value": "oracle reward:\n always chooses the best arm\n\npolicy reward:\n sometimes makes mistakes while learning\n\nregret =\n oracle cumulative reward\n - policy cumulative reward" } ] }, { "number": "23", "title": "Epsilon-Greedy: The Easiest Way to Explore", "tag": "Bandit", "description": "Epsilon-greedy works crudely, but is very clear. With probability `1 - ε` you choose the best known arm, and with probability `ε` you randomly try something else. This is a simple baseline that is often used as a starting point before smarter strategies.", "blocks": [ { "kind": "code", "label": "algorithm", "language": "python", "value": "if rand() < ε:\n action = random arm\nelse:\n action = argmax estimated_reward\n\nupdate estimate after observing reward" } ] }, { "number": "24", "title": "UCB: underexplored options get uncertainty bonus", "tag": "Bandit", "description": "Upper Confidence Bound chooses not just the arm with the best average reward, but the arm with the best “optimistic score”. If an option has been little explored, an exploration bonus is added to it. Therefore, UCB automatically balances learning and use without random rolls like epsilon-greedy.", "blocks": [ { "kind": "code", "label": "UCB intuition", "language": "python", "value": "score(arm) =\n empirical_mean(arm)\n + uncertainty_bonus(arm)\n\nif arm is poorly studied:\n bonus big\n\nif arm is studied well:\n bonus small" } ] }, { "number": "25", "title": "Thompson Sampling: Explore through Probabilistic Faith", "tag": "Bandit", "description": "Thompson Sampling stores the uncertainty distribution over the quality of each arm and samples from this belief at each step. An Arm with a potentially high reward and high uncertainty has a natural chance of being selected. This is one of the most practical bandit approaches.", "blocks": [ { "kind": "code", "label": "simplified diagram", "language": "sql", "value": "for each arm:\n sample θ_arm from posterior\n\naction =\n argmax sampled_θ_arm\n\nobserve reward\nupdate posterior" } ] }, { "number": "26", "title": "Contextual Bandits: the best action depends on the context", "tag": "Policy", "description": "In many products there is no one best arm for everyone. It is better to show different options to different users. Then bandit receives the user's context as input and learns a policy of the form `π(action | context)`. This is closer to personalization, advertising, recommendations and adaptive ranking.", "blocks": [ { "kind": "code", "label": "Diagram", "language": "sql", "value": "FROM MULTI-ARMED TO CONTEXTUAL\n\nsimple bandit:\n choose best arm overall\n\ncontextual bandit:\n user features x\n ↓\n choose arm, the best one for x\n\nexample:\n new user → onboarding A\n power user → onboarding B\n budget user→ offer C" }, { "kind": "list", "label": "typical domains", "items": [ "ads", "recommendations", "homepage modules", "we need neat action-probabilities logging logic" ] } ] } ] }, { "number": "06", "title": "A/B, Bandits AND Online Evaluation", "intro": "", "cards": [ { "number": "27", "title": "A/B testing vs bandits: not competitors, but different tools", "tag": "Evaluation", "description": "An A/B test is good when you need to reliably measure the effect of fixed options. Bandits are good when you need to adaptively collect rewards along the way, without showing weak options for too long. They are often combined in products: first A/B for pure measurement, then bandit for adaptive serving.", "blocks": [ { "kind": "table", "headers": [ "Approach", "Main goal", "Strength" ], "rows": [ [ "A/B", "measure the causal effect", "pure interpretation" ], [ "Bandit", "maximize reward while learning", "less losses on weak options" ], [ "Hybrid", "measure + adapt", "realistic food compromise" ] ] } ] }, { "number": "28", "title": "Offline evaluation is not equal to online success", "tag": "Evaluation", "description": "Forecasting can be fairly fairly validated by backtest. Causal - through experimental design and assumptions. Bandits and policy systems often run into the fact that offline logs are collected by the old policy, and the new policy will collect different data. Therefore, online validation is especially important here.", "blocks": [ { "kind": "list", "label": "main idea", "items": [ "offline is useful for weeding out weak ideas", "online determines the real product value", "do not confuse replay on old logs with true future behavior" ] } ] }, { "number": "29", "title": "Delayed feedback and non-stationarity", "tag": "Workflow", "description": "In many real-world systems, the reward does not come immediately, but after hours or days. Plus, the environment is changing: season, competitors, interface, economic situation, new user segments. This complicates forecasting, bandits, and post-launch causal assessment.", "blocks": [ { "kind": "code", "label": "what breaks the system", "language": "text", "value": "delayed feedback:\n action now\n reward tomorrow\n\nnon-stationarity:\n P(data) changes over time\n\npolicy feedback loop:\n the system itself changes user behavior" } ] } ] }, { "number": "07", "title": "How to Choose the Right Frame", "intro": "", "cards": [ { "number": "30", "title": "Decision Tree: forecasting, causal inference or bandit?", "tag": "Bridge", "description": "When a product problem is formulated vaguely, it is useful to quickly run it through one decision tree. Usually this is enough to avoid launching an erroneous project under the guise of “let’s just make an ML model.”", "blocks": [ { "kind": "code", "label": "Diagram", "language": "sql", "value": "QUICK SELECTION OF POSITION\n\nQuestion:\n you need to understand what will happen next,\n if we don't change anything?\n → Forecasting\n\nNeed to understand the effect of a change?\n → Causal inference\n\nYou need to select action online,\nAre we still learning?\n → Bandit / political learning\n\nDo you just need to recognize the class of an object?\n → regular supervised ML" } ] }, { "number": "31", "title": "Where is the place for reinforcement learning?", "tag": "Bridge", "description": "Bandits are often called “RL lite”. The difference is that bandit usually does not model long sequences of states and long-term consequences through state transitions. RL is needed where the action now affects what state you will be in next, and this is repeated along the chain.", "blocks": [ { "kind": "code", "label": "comparison", "language": "text", "value": "bandit:\n context → action → immediate reward\n\nRL:\n state → action → next state → ... → long-term return\n\nIf there are no long state dynamics, bandit is often simpler and more practical" } ] }, { "number": "32", "title": "The main idea of the volume", "tag": "Bridge", "description": "There is no one universal “ML model for business”. There are different classes of questions. If you need a forecast, build forecasting and think about the horizon and backtesting. If you want an intervention effect, think causal and assumptions. If you need an online selection of actions, go to bandits and regret. Correct setup saves months of useless optimization.", "blocks": [ { "kind": "list", "label": "short reminder", "items": [ "predict future → forecasting", "estimate impact → causal", "choose adaptively → bandit", "do not confuse these tasks under the word “ML”" ] } ] } ] } ] }