Complete taxonomy by operational goal
The most common failure mode for emerging data scientists is algorithm-first thinking — the assumption that ML architectures are themselves the solutions. In production systems engineering, operational goals dictate applied topics, and ML algorithms are merely the implementation primitives selected to satisfy those goals under specific mathematical and infrastructural constraints. Systems are classified here by what the module functionally achieves in its environment, then mapped down to the concrete tools used to build it.
| Operational goal | Applied topics | Functional rationale | Implementation mapping |
|---|---|---|---|
| Generative & Creative Synthesis |
| Constructing novel artifacts: Systems map high-dimensional latent representations into structured, domain-specific outputs — tokens, pixels, waveforms — that generalize beyond the training distribution to synthesize data that did not previously exist. | Autoregressive generationDiffusion models (score matching, Langevin dynamics)Variational autoencodersGANsCross-entropy optimizationLatent-space interpolation |
| Information Retrieval, Ranking & Personalization |
| Traversing latent spaces: Systems select the most relevant items from a massive catalog by projecting queries and entities into a shared semantic vector space, so relevance becomes a geometry problem rather than a scan. | Two-tower architecturesContrastive learning (InfoNCE, triplet loss)Approximate nearest-neighbour indices (HNSW, FAISS)Session-based sequence modelingMatrix factorization |
| Perception & Sensory Signal Understanding |
| Translating unstructured noise: Systems ingest continuous physical signals — photons, acoustic waves, LiDAR returns — and extract deterministic semantic attributes such as bounding boxes, phonemes, and spatial coordinates, while remaining robust to environmental noise and out-of-distribution input. | CNNs and Vision TransformersSpatial feature pyramidsBounding-box regression (IoU loss)Connectionist temporal classificationOut-of-distribution detection |
| Natural Language Understanding & Knowledge Extraction |
| Mapping symbolic intent: Systems parse the syntax, logic, and grammar of human-authored language into structured relational representations, deterministic schemas, or downstream execution payloads — the output is machine-actionable structure, not prose. | Encoder-only Transformers (BERT family)Sequence-to-sequence mappingConditional random fields for token classificationSelf-attention mechanismsMasked language modeling |
| Decision-Making & Closed-Loop Control |
| Dynamic policy execution: Systems evaluate partially observable environmental states and emit sequential action policies that maximize cumulative reward over a continuous operating horizon. The defining property is that the action changes the next observation, which no supervised loss can express. | Reinforcement learning (PPO, SAC, DDPG)Markov decision processesBellman equationsTrajectory optimizationImitation learningExplore/exploit heuristics |
| Graph & Topology Mining |
| Exploiting non-Euclidean structure: Systems model environments where the functional logic is carried by relational connectivity, edge weights, and node neighbourhoods rather than by position in a matrix grid — so the structural prior is adjacency, not locality. | Graph neural networks (GCN, GAT, GraphSAGE)Message-passing architecturesSpectral clusteringDeepWalk / Node2Vec embeddingsLink-prediction heuristics |
| Causal Inference & Counterfactual Evaluation |
| Estimating treatment effects: Systems isolate cause from statistical confounding, evaluating interventions that were never observed. Predictive accuracy is explicitly not the objective here — identification comes from the design, and the estimand is an effect with an interval. | Do-calculusInverse probability of treatment weightingMeta-learners (S-, T-, X-learner)Double machine learningInstrumental variables |
| Surrogate Modeling & Digital Twins |
| Emulating complex dynamic systems: Systems approximate a computationally expensive or non-linear physical process by mapping sequential inputs to predicted future states, standing in for a simulator that is too slow to query in the loop. Non-stationarity is the binding constraint, which is why evaluation must be walk-forward. | Physics-informed neural networksNeural ODEsLSTMs and Transformers with temporal attentionAutoregressive forecastingExpanding-window / walk-forward evaluation protocols |
Scroll the table horizontally to see all four columns.
Where this site goes deepest
Time-Series Forecasting
Observations are ordered and dependent, which breaks the i.i.d. assumption every other supervised setting rests on. The signal usually decomposes into trend, seasonality, and autocorrelated noise, and the label is simply the series' own future value — so "more data" often means a longer history of the same process, not more independent examples. The binding constraint is that you must never let information from the future leak backwards, which rules out ordinary shuffled cross-validation outright.
Baseline to beat: Seasonal naive — last week's same weekday. Report it alongside every model you propose; a surprising share of production forecasters do not beat it.
Which model, and whyAnomaly Detection
The positive class is rare (often well under 1%), frequently unlabeled, and the interesting anomalies are the ones you have never seen — so the problem is usually "model normal and measure deviation", not "classify". Accuracy is meaningless at this base rate, and the real constraint is almost always the alert budget: how many false positives per day a human team will tolerate before they stop reading the alerts.
Baseline to beat: A robust z-score or IQR rule on the single most informative signal. It is trivially explainable, and beating it by enough to justify a model is the actual bar.
Which model, and whyOptimization
The deliverable is a decision, not a prediction: choose the assignment, schedule, price, or allocation that maximizes an objective subject to hard constraints. Feasibility usually matters more than accuracy — a slightly worse feasible plan beats an excellent infeasible one — and the objective is frequently non-differentiable or only observable by trying it.
Baseline to beat: A greedy heuristic or a linear program. Both are fast, explainable, and give a provable feasible solution — and an ML approach that cannot beat them is not worth its operational cost.
Which model, and whyThe wider field, for contrast
Included because the interesting part of a category verdict is where it flips. Linear regression is a middling forecaster and the default estimator in causal inference; reinforcement learning is the default for control and the wrong tool for detection. Seeing the same four families ranked differently across nine problems is the point.
Recommendation & Ranking
Feedback is implicit and missing-not-at-random: you observe what was shown and clicked, never what would have been clicked had it been shown. The output is an ordering rather than a value, so the loss is pairwise or listwise, and the system is a feedback loop — today's model shapes tomorrow's training data.
Baseline to beat: Most-popular-by-segment, and item-to-item collaborative filtering.
Which model, and whyNatural Language
Discrete tokens, long-range dependency, and enormous quantities of unlabeled text. The defining property is that self-supervised pretraining transfers: a model trained to predict the next token learns representations that carry to nearly every downstream task, which inverted the field's economics.
Baseline to beat: TF-IDF plus logistic regression — still the bar for narrow classification.
Which model, and whyComputer Vision
Pixels carry strong spatial structure: nearby pixels are correlated, and an object is the same object wherever it appears. Those two facts — locality and translation invariance — are the priors that make the problem tractable at all, and they are what convolution encodes directly.
Baseline to beat: A frozen pretrained backbone with a linear probe on top.
Which model, and whyCausal Inference & Uplift
The target is a counterfactual that is never observed: what would have happened under the other treatment. Predictive accuracy is not the goal and can actively mislead — a model that perfectly predicts who churns says nothing about who an intervention would save. Identification comes from design (randomization, instruments, discontinuities), not from model capacity.
Baseline to beat: A randomized experiment with a difference-in-means. Nothing beats a clean A/B test.
Which model, and whyRisk & Fraud
Tabular, extremely imbalanced, adversarial, and regulated all at once. The adversary adapts to your model, labels arrive late (a chargeback lands months after the transaction), and in credit decisions you may be legally required to explain any adverse outcome — which rules out anything you cannot interpret.
Baseline to beat: Expert-authored rules. They are strong, explainable, and the incumbent a model must beat.
Which model, and whyControl & Operations
Physical or logistical systems with state, dynamics, hard constraints, and real consequences for failure. Actions change the state, safety is non-negotiable, and the objective spans a horizon rather than a single step — which is exactly where prediction stops being sufficient.
Baseline to beat: A classical PID controller or a mixed-integer program over a point forecast.
Which model, and why