Start from the problem, not the model

What are you trying to produce?

The most general question there is, and the only one that reliably narrows the field before you know anything else. The shape of the output — a number, a label, a sequence, new data, or a decision — determines the loss, and the loss determines the family.

• A number or a score

  Do you have labelled examples of the number you want?

  Supervision is the next binding constraint. With labels this is regression; without them you are describing the data instead, which is a different family entirely.

  • Yes — I have labels

    Do you need to explain the prediction to someone?

    Interpretability is a hard requirement in regulated and high-stakes settings, and it narrows the field before accuracy does. It is a constraint, not a preference.

    • Yes — it must be explainable

      Start with the linear family.

      A regularized linear model gives coefficients you can defend, calibrated intervals, and a baseline that a more complex model has to beat before it earns its operational cost.

      Linear Models

    • No — accuracy is what matters

      Go to gradient boosting.

      On tabular data, boosted trees remain the strongest default. Reach past them only when the data stops being tabular.

      Gradient Boosting

  • No labels

    Are you looking for axes, or for groups?

    Without a target the only options are describing variation (a low-rank basis) or partitioning (clusters). Both are unsupervised, but they answer different questions.

    Structure & Clustering

• A category or a group

  How rare is the class you care about?

  Balanced classification and rare-event detection look identical on paper and behave nothing alike. Below roughly 1% positives, accuracy stops meaning anything and the problem becomes "model normal, measure deviation".

  • Balanced — both classes are common

    Is your data tabular, or perceptual?

    Tabular data still belongs to gradient boosting; images, audio, and text belong to learned representations. This is the most reliable modality split in applied ML.

    Trees & Ensembles

  • Rare — under ~1%

    Do you have labelled anomalies, or only normal data?

    Almost always only normal data, and the interesting anomalies are ones never seen before — which is why this is a modelling-normality problem rather than a classification one.

    Anomaly Detection

• A sequence over time

  One series, or many related series?

  This single question separates classical forecasting from deep forecasting more reliably than anything else. Cross-series structure is the only thing that pays for a deep model; without it you are fitting a large model to a small problem.

  • One short series

    Does it have clear trend and seasonality?

    On a few hundred points the structure is low-dimensional and a stated model beats a learned one. Start at the seasonal-naive baseline and make anything fancier earn its place.

    Classical Time Series

  • Many, long, or coupled series

    Are the series linked by a known structure?

    If the relationships are known — a road network, a grid, a supply chain — a graph prior encodes them directly. If they are merely numerous, a global sequence model shares statistical strength without needing the topology.

    Time-Series Forecasting

    • Yes — the topology is known

      Encode the relationships directly.

      When you already know which series influence which — a road network, a power grid, a supply chain — a graph prior states that structure instead of making the model rediscover it from data.

      Graph architectures

    • No — just a great many of them

      Train one global sequence model across all of them.

      Thousands of related series share statistical strength even without a known topology. This is the case where a deep model genuinely beats per-series classical fits, and the only one where it reliably does.

      Sequence architectures

• New data or content

  Do you need an exact likelihood, or just good samples?

  The central trade-off of generative modelling. Exact density, fast sampling, and stable training cannot all be had at once, and which one you give up defines the architecture.

  • I need exact likelihood

    Is the data sequential, or continuous?

    The chain rule gives exact likelihood for free on sequences; on continuous data you need an invertible map. That is the whole reason both autoregressive models and normalizing flows exist.

    Latent-Variable Models

  • Samples matter more than density

    Can you afford many steps at sampling time?

    Diffusion buys quality and training stability by spending many forward passes per sample. If inference latency binds, a one-shot generator is the trade you make.

    Diffusion

• A decision under feedback

  Does your action change what you observe next?

  The dividing line between prediction and control. If actions change the state, you have a sequential decision problem and no supervised loss can express the objective. If they do not, you have a much simpler bandit.

  • Yes — there is state and delayed reward

    Are your actions discrete, or continuous?

    Value-based methods take a max over actions, which is only tractable when actions are discrete. Continuous action spaces force an actor-critic formulation — this is the fork inside RL.

    Reinforcement Learning

    • Discrete — a finite menu

      Learn the value of each action.

      Taking a max over actions is only tractable when there are finitely many, which is exactly the assumption value-based methods rest on.

      Value-based methods

    • Continuous — a real-valued setting

      Parameterize the policy directly.

      Greedy maximization over a continuous action space is not tractable, which forces an actor-critic formulation — the fork that defines this half of the category.

      Continuous control

  • There are hard constraints to satisfy

    Must every answer be feasible, not just good?

    Once feasibility is non-negotiable — a schedule that must fit, a budget that must balance — a learned model that cannot guarantee constraint satisfaction belongs inside the solver rather than replacing it. That reframing is what separates optimization from prediction.

    Optimization

  • No — each choice is independent

    Do you have context features for each decision?

    A stateless repeated choice is a bandit, not an MDP, and reaching for full RL here adds instability for nothing. This is the most common correct answer in production, and the most commonly over-engineered.

    Multi-Armed Bandits