Forecasting, Causal Inference & Bandits

A lot of the confusion in applied ML starts not with the model, but with the wrong question. `Forecasting` asks: what is most likely to happen next? `Causal inference` asks: what will change if we intervene? `Bandit/policy` asks: what is better to choose right now, while we are still learning?

Classic supervised ML often assumes that the examples are independent and the distribution does not drift too quickly. In a time series, neighboring points depend on each other in time. In causal, we are not just interested in the observed correlation, but in the effect of the intervention. In bandits, the data collection policy itself affects future data.

Sometimes the model needs to predict a future value. Sometimes estimate the effect size of an action. Sometimes choose the action with the best expected reward. If you mix these three outputs, you can get a technically neat, but productually useless system.

To speak the same language with researchers and analysts, it is enough to remember a few central terms. They constantly pop up in articles, discussions of experiments, and product reviews.

A time series is rarely just a list of numbers. It is typically a mixture of slow baseline changes, repeating seasonal cycles, random noise, and occasional external shocks. A conversation about forecasting often begins with the decomposition of these components.

If today's value is similar to yesterday's or to the value a week ago, then there is auto-dependence in the series. This is why lags and rolling statistics are so important: they give the model direct access to the past state of the system.

Forecasting one step ahead and forecasting 30 steps ahead are not the same task. The further the horizon, the higher the uncertainty and the more errors accumulate. In multi-step systems, you need to decide in advance whether you will build one general multi-horizon forecast or repeat a single-step forecast along the chain.

The usual random split for time series is almost always wrong. The future should not be included in the training of the model that supposedly predicts this future. Therefore, they use time-based split, rolling origin evaluation and `TimeSeriesSplit`-like schemes, where train is always strictly before validation.

The most common failure of a forecasting project is not the “wrong model”, but an unnoticeable leak of the future. Any rolling mean, target encoding, cumulative metric or exogenous signal should be considered as if you are actually living at time `t` and do not yet know the future.

In forecasting it is very easy to build a complex model that is worse than the simplest baseline. Therefore, they almost always consider at least `last value`, `seasonal naive` or `drift`. If you don't beat these primitives, the project has not yet reached the real ML win.

In practice, forecasting often follows two paths. First: specialized time-series models like ETS / ARIMA, which explicitly describe the dynamics of the series. Second: feature-based ML, where you build lags, rolling features and exogenous features, and then train gradient boosting or another supervised model.

Very often forecasting is solved not by a separate “magic” model, but by ordinary supervised ML. You create lags, calendar features, prices, stocks, balances, weather data and turn each time point into a table row. Then you can learn boosting, a linear model or a neural network.

Different forecasting metrics penalize the model in different ways. RMSE hits harder for large errors, MAE is more robust, MAPE breaks down around zero, and sMAPE partially corrects this defect. Therefore, one cannot say “the model is better” without specifying by what metric and on what horizon.

The usual predictive model answers the question “who buys more often?” well, but it still does not answer the question “what will happen if we send a promotional code?”. A model can find strong correlations and yet not know the true effect of an action at all. Causal inference is needed exactly where we are interested in the result of the intervention.

The basic causal intuition is simple: the same person or object has at least two potential outcomes - with treatment and without treatment. The problem is that in reality we only see one of them. The second world remains a counterfactual, and that is why causal inference is more difficult than ordinary prediction.

If treatment is not obtained by chance, but because the object already differs in important ways, a simple comparison of the treated and untreated groups will be biased. This is confounding. Usually, causal work is largely a fight against it.

Randomization makes the treatment independent of hidden factors on the sample average. This is why A/B tests are so valuable: they eliminate much of the confounding without complex math. If a good experiment can be done, this is usually the best way to estimate a causal effect.

If an experiment is not possible or too expensive, causal inference attempts to approach it through research design. There is no one universal method: matching, regression adjustment, inverse propensity weighting, difference-in-differences, regression discontinuity and instrumental variables solve different types of bias.

The average effect is useful, but heterogeneity is often more important in products. One segment benefits from treatment, another is neutral, and the third even gets worse. Hence the language of `CATE`, `uplift modeling`, `treatment heterogeneity` and personalized interventions.

Even neat teams regularly make mistakes in causal statements. Most often, this is either an incorrect assumption about parallel trends / ignorability, or post-treatment leakage, or an overly optimistic transfer of the effect from one environment to another.

The Bandit problem occurs when you need to choose an action right now, but you are not yet sure which action is best. If you always choose the current leader, you may miss the best option. If you experiment too much, you lose the reward today. It is this compromise that is called exploration vs exploitation.

In bandits, it is important not only how much reward you collected, but also how much you lost relative to the ideal oracle, which always knows the best action. This difference is regret. A good bandit policy makes regret grow slowly as experience accumulates.

Epsilon-greedy works crudely, but is very clear. With probability `1 - ε` you choose the best known arm, and with probability `ε` you randomly try something else. This is a simple baseline that is often used as a starting point before smarter strategies.

Upper Confidence Bound chooses not just the arm with the best average reward, but the arm with the best “optimistic score”. If an option has been little explored, an exploration bonus is added to it. Therefore, UCB automatically balances learning and use without random rolls like epsilon-greedy.

Thompson Sampling stores the uncertainty distribution over the quality of each arm and samples from this belief at each step. An Arm with a potentially high reward and high uncertainty has a natural chance of being selected. This is one of the most practical bandit approaches.

In many products there is no one best arm for everyone. It is better to show different options to different users. Then bandit receives the user's context as input and learns a policy of the form `π(action | context)`. This is closer to personalization, advertising, recommendations and adaptive ranking.

An A/B test is good when you need to reliably measure the effect of fixed options. Bandits are good when you need to adaptively collect rewards along the way, without showing weak options for too long. They are often combined in products: first A/B for pure measurement, then bandit for adaptive serving.

Forecasting can be fairly fairly validated by backtest. Causal - through experimental design and assumptions. Bandits and policy systems often run into the fact that offline logs are collected by the old policy, and the new policy will collect different data. Therefore, online validation is especially important here.

In many real-world systems, the reward does not come immediately, but after hours or days. Plus, the environment is changing: season, competitors, interface, economic situation, new user segments. This complicates forecasting, bandits, and post-launch causal assessment.

When a product problem is formulated vaguely, it is useful to quickly run it through one decision tree. Usually this is enough to avoid launching an erroneous project under the guise of “let’s just make an ML model.”

Bandits are often called “RL lite”. The difference is that bandit usually does not model long sequences of states and long-term consequences through state transitions. RL is needed where the action now affects what state you will be in next, and this is repeated along the chain.

There is no one universal “ML model for business”. There are different classes of questions. If you need a forecast, build forecasting and think about the horizon and backtesting. If you want an intervention effect, think causal and assumptions. If you need an online selection of actions, go to bandits and regret. Correct setup saves months of useless optimization.