Computer Vision

Slide a small learned filter across the input and record how strongly it responds at each position. Two assumptions are baked into that operation and they are the whole reason it works: nearby inputs are related (locality), and a feature worth detecting in one place is worth detecting everywhere (translation equivariance, via weight sharing). Stack these layers and the receptive field widens, so early layers see edges and later ones see objects.

A classifier answers "what is this image". A detector has to answer "what is here, and where, and how many" — and the last part is the hard one, because a network produces a fixed-size output and an image contains a variable number of objects. Every design resolves this the same way: predict far more boxes than there could be, have almost all of them predict "nothing", and clean up the duplicates afterwards. That structure explains the two facts that dominate the field. The class imbalance is extreme, because background outnumbers objects by orders of magnitude. And the deduplication step — non-maximum suppression — is a hand-written greedy algorithm bolted to the outside of a learned system, which is exactly the kind of seam that later designs try to remove.

Instead of asking a stack of layers to learn the mapping H(x) directly, ask it to learn only the correction F(x) = H(x) - x and add the input back: y = F(x) + x. If the best thing a block can do is nothing, it only has to push its weights toward zero, which is easy; a plain stack has to learn the identity map through several nonlinear layers, which turns out to be hard. That is the whole idea, and its consequence was surprising: before it, making a network deeper made it train worse, not just generalise worse. After it, hundreds of layers train with ordinary SGD, and the shortcut became the default way to wire any deep network.

Downsampling is how a network learns WHAT is in an image, and it is exactly what destroys WHERE it is. A U-Net refuses to choose. A contracting path (convolution, then downsample, repeated) trades resolution for context until the bottleneck sees the whole scene. An expanding path (upsample, then convolution) climbs back to input resolution. And at every scale the encoder feature map is CONCATENATED onto the decoder feature map, handing the decoder the fine spatial detail the pooling threw away, so it only has to decide what each pixel is rather than reconstruct where the boundary was. The output is a class distribution at every pixel, which is why it is called dense prediction. Concatenation is the point: ResNet adds its skip, which forces the two sources to share channels and be summed blindly, whereas a U-Net stacks them and lets the next convolution learn how much of each to trust.

Cut the image into a grid of 16-by-16 patches, flatten each one into a vector, and feed the resulting sequence to a plain text transformer with nothing changed. The model is told nothing about two-dimensional structure — not locality, not translation equivariance, not scale — beyond a learned position embedding per patch. That sounds like it should not work, and on a hundred thousand images it does not: below a certain data scale a convolutional network wins comfortably, because the biases a CNN hardcodes are true and free. Above that scale the transformer learns better biases than the ones we knew to build in, and overtakes.

Train a forger and a detective at the same time. The forger produces samples, the detective tries to tell them from real data, and each improves against the other. Neither ever sees an explicit description of what makes data realistic — the forger only ever learns from the detective’s mistakes. That is the appeal, because it means no likelihood needs to be defined and no reconstruction distance has to be chosen, which is why GAN images were sharp when everything else was blurry. It is also the problem: the target is moving, so there is nothing that decreases monotonically and no loss value that tells you how well it is going.

A plain GAN’s discriminator answers a classification question, and once it can answer it confidently it stops being informative — the gradient it hands back vanishes exactly when the generator most needs guidance. Replace it with a critic that scores rather than classifies, constrain how fast that score can change, and the answer becomes a distance that stays meaningful however far apart the two distributions are. Separately and independently: tell both networks what they are looking at. A generator given a class label learns one conditional distribution per label rather than one blurred mixture, and the critic can then check that the sample matches its label rather than merely looking real.

Generating an image in one step is hard; removing a little noise from a slightly noisy image is easy. So define a process that gradually destroys data into pure noise over a thousand steps, train one network to undo a single step of it, and then run that network a thousand times starting from noise. Nothing about any individual step is difficult, which is why this trains stably where a GAN does not — and the price is paid exactly where you would expect, at inference, where a thousand sequential network evaluations produce one sample.

Stop thinking about a thousand discrete denoising steps and think about a continuous path from noise to data. At any point along that path there is a direction that leads toward the data distribution — the score, or equivalently a velocity — and a network can be trained to predict it by regression. Once you have that direction field, generating a sample is solving a differential equation, which means every numerical-integration technique becomes available. Flow matching then asks the obvious follow-up: if the path is ours to choose, why not choose a straight one? A straight path can be traversed in very few large steps, which is the whole reason modern samplers need dozens rather than thousands.

Nothing here specifies what an embedding should be. It specifies only that paired things should end up closer than unpaired things, and the geometry falls out of enforcing that over millions of pairs. This is why the technique is defined almost entirely by its data: the encoder is ordinary, and the consequential choices are what counts as a positive pair, where the negatives come from, and the temperature that decides how hard the loss pushes on the closest wrong answer. Get the pairs right and a small encoder beats a much larger one. Get them wrong and the embeddings look fine on held-out pairs from the same distribution while failing on everything real, because the model learned to solve the sampling artefact rather than the task.

Fine-tuning a large model means changing every weight, which means storing a gradient and two optimizer moments for every weight — several times the model's own size in memory before a single activation is saved. The observation behind LoRA is that the CHANGE a fine-tune makes is far simpler than the model it changes: adapting a general model to legal drafting or to one company's support tone is not learning language again, it is nudging behaviour the model already has. So freeze the pretrained weight and learn a low-rank correction beside it, W0 + BA with an inner dimension of eight or sixteen instead of four thousand. Two consequences fall straight out. Training memory collapses, because optimizer state is proportional to trainable parameters and there are now a thousand times fewer. And the result is a small file that means nothing on its own, which is a feature — one frozen base can serve hundreds of tenants, each with their own adapter, if you are willing to keep it unmerged. The failure mode is the mirror image of the premise: when the task really does require capability the base does not have, a low-rank update cannot express it, and no amount of tuning reveals that the ceiling is structural rather than statistical.

Imagine every data point is a voter and every data point is also a candidate. Each voter tells each candidate how much it would like that candidate to represent it, measured against its best alternative — that is the responsibility. Each candidate answers with how available it is, which depends on how much support it is already collecting from everyone else — that is the availability. The two messages are passed back and forth until the votes settle, and the points that end up voting for themselves are the exemplars. Nobody told the algorithm how many clusters to find; that falls out of how reluctant each point is to stand as a candidate, a number called the preference. The leaders are real data points, not averages, which means the only thing the method ever needs is a table of how similar each pair is — and that table can come from edit distance, from correlation, from anything that can be computed between two items.

You have a thousand audio frames and a nine-character transcript, and nobody has told you which frames belong to which character. Rather than guess an alignment, add a blank symbol meaning "emit nothing here" and score every alignment that collapses to the transcript, summing them all. There are exponentially many, but because the alignment can only move forward, the sum factorizes into a two-pass dynamic program that computes it exactly.

Every position looks at every other position and decides, from content alone, which ones matter to it. A convolution asserts in advance that neighbours are relevant; recurrence asserts that recency is. Attention asserts neither — it learns the routing. That is why it transfers across modalities, and also why it needs far more data than architectures whose prior is built in: it has to learn from scratch what the others assume.

Force the data through a narrow layer and demand that it come out intact. Because the bottleneck cannot carry everything, the network has to discover what is worth keeping — and whatever survives that squeeze is a representation. The corollary is the useful part: an input the network cannot rebuild is an input unlike anything it learned to compress, which is exactly what an anomaly is.

Once a model has turned a million documents into a million vectors, answering a query means finding the nearest few — and comparing against all million is too slow. So give up exactness: build a structure that looks at a few thousand candidates instead of a million and usually finds the same answer. The word "usually" is the entire subject. An ANN index does not fail by being slow, it fails by silently returning the second-best neighbour, and how often it does so is a dial you set rather than a property you measure once.

Two separate encoders — one for the query, one for the item — trained so that relevant pairs land close together in a shared vector space. The separation is the entire architectural point: because the item tower never sees the query, every item embedding can be computed offline and indexed. Retrieval then becomes nearest-neighbour search rather than scoring millions of candidates one at a time. A cross-encoder is more accurate and structurally cannot do this.

An autoencoder that is forced to be probabilistic: instead of mapping an input to a single point in latent space, the encoder outputs a small Gaussian, and the decoder must reconstruct from a sample of it. Because nearby latent points now decode to similar outputs, the latent space becomes continuous rather than a scatter of memorized codes — which is what makes sampling from it produce something new rather than noise.

Do not build a model of the world. Keep every example you have seen, and when a new point arrives, find the handful that look most like it and copy their answer — a majority vote for a class, an average for a number. Everything hard about the method is hidden in "look most like it": the distance function is the model, and choosing it badly is a modelling error that no amount of tuning k will repair. The trade is stark and worth stating up front: training costs nothing, and inference costs everything.

Of all the hyperplanes that separate two classes, take the one that sits as far as possible from the nearest points on either side. That clearance is the margin, and maximizing it is a bet worth stating plainly: a boundary with room around it is more likely to survive new data than one that squeaks between training points. Two things follow. Only the points on or inside the margin — the support vectors — affect the answer, so most of the training set could be deleted without changing the model. And because the solution depends on the data only through inner products, replacing that inner product with a kernel produces a nonlinear boundary without ever computing the coordinates it lives in.

One Gaussian describes a single blob; real data is usually several, so describe it as a weighted sum of Gaussians and let the fit decide how much of each. The obstacle is that you never observe which component produced a point — that assignment is the latent variable — so you alternate. Given the current components, compute how responsible each is for each point. Given those responsibilities, refit each component to the points it owns, weighted by ownership. The defining feature is that ownership is fractional: a point between two clusters belongs 60/40 rather than being forced into one. k-means is this same loop with the responsibilities rounded to 0 and 1 and every covariance forced spherical and equal, which is exactly why k-means cannot represent an elongated or overlapping cluster and this can.

Data with many correlated columns does not really occupy all those dimensions — it sits near a lower-dimensional plane inside them. PCA finds that plane: the direction along which the points spread out most, then the direction of most remaining spread perpendicular to it, and so on. Keep the first few and you have a compressed description that loses very little. The catch is in what "loses very little" means, and it is worth being suspicious of: PCA measures loss as squared distance, so it keeps whatever varies most. Variance is not the same as information, and the direction that separates two classes can easily be one of the ones you just discarded.

Summarize the data with k representative points. Start with k guesses, assign every observation to its nearest guess, then move each guess to the centre of what it just captured, and repeat until nothing moves. The objective it descends is the total squared distance from points to their own centre, and the two steps alternate between the two things that objective depends on — which assignment, and which centres. What makes it so widely used is that both steps are trivial and it converges quickly. What makes it so widely misused is that it always returns exactly k clusters, whether or not the data has any.

Stop thinking about distance to a centre and start thinking about connectivity. Build a graph where nearby points are joined, then look for a way to cut it into pieces that severs as few connections as possible. Two crescents that interlock have no separating hyperplane and no useful centroids, but they are two obviously distinct connected regions, and cutting the graph finds them. The trick that makes this computable is that the best cut, once relaxed from a hard yes-or-no assignment to a continuous one, is given by the smallest eigenvectors of the graph Laplacian — so a combinatorial problem becomes a linear-algebra one. Those eigenvectors are new coordinates in which the crescents have become blobs, and then k-means finishes the job.

Separate what the system is doing from what you managed to measure. The state is the thing you care about — a position, a level, a trend — and it evolves by its own rules. The observation is a noisy, possibly incomplete look at it. Then run two steps forever. Predict: push the state forward and let the uncertainty grow, because time passing makes you less sure. Update: take the measurement and move the estimate toward it, by an amount that depends on which you trust more. That trade-off is the whole filter, and the Kalman gain is literally its arithmetic — when your prediction is uncertain and the sensor is good, the gain is near one and you believe the sensor; when the reverse, you barely move.

Start from a simple distribution you can sample and evaluate — a standard Gaussian — and push it through an invertible function. The change-of-variables formula says exactly what happens to the density: it is the base density at the preimage, corrected by how much the transformation stretched or compressed space there. So if you can build a flexible invertible function whose local volume change you can compute cheaply, you get an exact likelihood for free. The entire field is a search for such functions, and every architecture in it is a different answer to the same question: how do you keep a Jacobian determinant tractable without making the map trivial?

A causal model sees only the left context because it has to: the chain rule imposes an order. If you are not going to generate, that constraint buys nothing — a classifier reading a sentence has the whole sentence. So hide fifteen percent of the tokens, let every position attend in both directions, and predict what was hidden. The representations are strictly better for understanding tasks because each token is informed by what follows it as well as what precedes it. What you give up is the ability to generate and the ability to compute a likelihood, and both losses follow from the same fact.

A pretrained model can produce a plausible continuation but has no notion of which of two plausible continuations anyone would rather receive, and the reason is structural: next-token likelihood rewards imitating the training distribution, which contains excellent and terrible text in unknown proportion. Preferences are the missing supervision, and they are cheap in a specific form — people are unreliable at scoring a response out of ten and fairly reliable at saying which of two they prefer. So collect comparisons, fit something that explains them, and push the model toward what wins. Two routes from there. Classic RLHF fits a reward model to the comparisons and then optimizes the policy against it with reinforcement learning, held near the original by a KL penalty. DPO observes that the optimal policy under that penalty has a closed form, inverts it to express the reward in terms of the policy itself, and lands on a logistic loss over preference pairs with no reward model and no sampling. The idea that makes both work is also the one that makes them fail: because only differences are ever observed, the optimization has a direction it can run in forever, and the KL term is the only thing holding it. Release that and you get a model that scores superbly and is worse.

Tabular Q-learning stores one number per state-action pair, which means it cannot play a game whose screen has more configurations than there are atoms in anything. Replace the table with a network that maps a state to a vector of action values and the method generalizes: states that look alike get similar values without being visited separately. That substitution is the entire idea, and it breaks the convergence proof immediately. The proof relied on each update touching exactly one cell; a network update moves every state at once, including the state the target was just computed from, so the agent is chasing a target that moves because it moved. DQN makes this work with two blunt interventions. Experience goes into a large replay buffer and minibatches are drawn from it at random, which breaks the temporal correlation that makes consecutive samples nearly identical. And the target is computed from a separate copy of the network that is frozen for thousands of steps, which stops the target from moving every time the estimate does. Neither is a fix. They convert a method that reliably diverges into one that usually does not, and the residual failure is not an error message — it is a Q-value curve that climbs smoothly to a thousand while the agent gets worse. The right time to reach for DQN is when actions are discrete, a simulator can supply tens of millions of transitions, and a policy-gradient method is not a better fit. That is a narrower set of conditions than its fame suggests.

DQN derives its policy from an argmax over the action values, which requires enumerating the actions. For a continuous action space that is not merely expensive, it does not exist. The deterministic policy gradient resolves this with a substitution that is obvious in hindsight: if the argmax cannot be computed, train a network to approximate it. So there are two networks — a critic Q(s, a) fitted by the usual Bellman regression, and an actor that outputs a single action and is trained to maximize the critic's output at that action. The actor's gradient is the critic's gradient with respect to the action, chained back through the actor, which means the critic must be differentiable in the action and the actor is climbing a surface the critic defines. That last clause is the whole story of this method's difficulty. The actor is an optimizer pointed at a learned function, and a learned function has errors — sharp spurious peaks in regions where little data exists. The actor will find them, because finding maxima is exactly its job, and it will then take the action at the peak, collect data showing the peak is not real, and the critic will correct it while the actor moves to the next one. DDPG does this and is famously brittle. TD3 adds three targeted countermeasures: two critics with the smaller value taken, so overestimation is replaced by deliberate pessimism; delayed actor updates, so the actor climbs a critic that has had time to settle; and noise added to the target action, which smooths the surface so a spike one action-width wide cannot be exploited. None of the three is clever. All three are aimed at the same failure.

Every method so far has treated exploration as something bolted on — an epsilon, a noise scale, an entropy bonus tuned until the run stops collapsing. SAC changes the objective instead: maximize return plus the entropy of the policy, weighted by a temperature. The agent is now paid to remain uncertain, and that single change does a surprising amount of work. Exploration is no longer a schedule to tune, because a policy that narrows onto one action loses entropy and therefore loses objective value; the exploration is whatever the return-versus-uncertainty trade says it should be, state by state. The value function changes shape too. The max over actions in the Bellman equation becomes a log-sum-exp — a soft maximum — so a state with several near-equally-good actions is worth more than a state with one, which is a genuinely different and often better notion of value. And because the policy is stochastic, the actor can be trained by reparameterizing the sample rather than by a score-function estimator, which means the gradient flows through the sampled action into the critic and carries far less variance than REINFORCE-style estimation. What remains from TD3 is the pessimism: two critics with the minimum taken, because an actor is still an optimizer pointed at a learned surface. What goes away is the target actor, the exploration noise schedule, and most of the brittleness. The temperature is the one new knob, and it is usually tuned automatically against a target entropy, which is why SAC in practice has fewer things to get wrong than anything else in this branch.