What it actually does
A classifier answers "what is this image". A detector has to answer "what is here, and where, and how many" — and the last part is the hard one, because a network produces a fixed-size output and an image contains a variable number of objects. Every design resolves this the same way: predict far more boxes than there could be, have almost all of them predict "nothing", and clean up the duplicates afterwards. That structure explains the two facts that dominate the field. The class imbalance is extreme, because background outnumbers objects by orders of magnitude. And the deduplication step — non-maximum suppression — is a hand-written greedy algorithm bolted to the outside of a learned system, which is exactly the kind of seam that later designs try to remove.
- Objects are well described by axis-aligned rectangles — false for rotated, articulated, or heavily occluded objects, which is why oriented and segmentation-based variants exist
- The anchor set or query count covers the object scales and shapes present, since anything outside it cannot be assigned a positive target
- Objects do not overlap so heavily that suppression cannot separate them, which crowded scenes routinely violate
- The class set is closed: an object of an unseen class is either missed or misclassified, never flagged as unknown
- Training and deployment share a visual domain, since a detector transfers worse across domains than a classifier does
Loss function
Classify every candidate and, for the ones that matched an object, also correct its box. The indicator is the part worth pausing on: background candidates get no box loss at all, because there is no ground-truth box for them and asking the network to predict one would be asking it to fit noise. The normalization by positive count matters for the same reason it usually does — without it an image with twenty objects produces a twenty-times-larger gradient than one with a single object, and the model quietly learns to care more about crowded scenes.
- predicted class distribution for candidate i, including a background class
- predicted box offsets — never absolute coordinates, always a correction to a reference
- the box loss applies only to positives; there is no correct box for background, so regressing one is meaningless
- the count of positive matches, used to normalize — otherwise the loss scales with how many objects happen to be present
How it is minimized
Joint SGD on classification and box regression, with an assignment rule matching predictions to ground truth and non-maximum suppression at inference
- intersection over union — the assignment rule during training, the evaluation metric, and the deduplication criterion, all at once
- boxes already kept, in descending score order; greedy suppression walks the list once
- the suppression threshold, and a genuine accuracy-versus-crowding trade rather than a technicality
- this is not learned — it is a hand-written greedy rule applied after the network, which is the seam DETR removes
Three decisions define a detector, and only one of them is the network. First, assignment: which predictions are responsible for which objects, usually by IoU against a set of anchor boxes, or by a bipartite matching in set-based designs. Get this wrong and the supervision is wrong regardless of the architecture. Second, the imbalance: with tens of thousands of candidates and a handful of objects, an unweighted cross-entropy is dominated by easy background and the model learns to predict nothing — which is what focal loss addresses, by down-weighting examples the model already gets right so the rare hard ones survive into the gradient. Third, deduplication: the network emits many overlapping boxes for one object, and greedy non-maximum suppression removes them at inference. That last step is not differentiable, not learned, and has its own failure mode on crowded scenes, which is precisely why set-prediction designs replace it with a matching that is part of training.
IoU assignment thresholds
What counts as a positive, a negative, or ignored during training. The most consequential setting in a detector and the least discussed (positive above 0.5-0.7, negative below 0.3-0.4, ignored in between)
NMS threshold
How much overlap is allowed among kept boxes. Lower removes duplicates and also removes genuinely overlapping objects (0.45-0.6, tuned against the crowding in the target domain)
anchor design
Scales and aspect ratios of the reference boxes. Mismatched anchors cap recall before training starts, which no amount of training fixes
focal loss gamma
How hard to down-weight easy examples. The mechanism that makes one-stage detectors trainable against the background imbalance (gamma 2.0, alpha 0.25)
score threshold
Confidence cut before NMS. A pure precision-recall dial applied at inference, not a model parameter
lambda (loss balance)
Weight on the box term relative to classification. Sensitive to the box parameterization, which is why IoU-based losses are more robust here
Training is ordinary SGD and converges unremarkably; the failures are about the surrounding structure. The dominant one is the imbalance: without focal loss or hard-negative mining, a one-stage detector trained with plain cross-entropy collapses to predicting background everywhere, and the loss looks like it is falling the whole time. Anchor mismatch is the quiet one — if no anchor overlaps an object shape by enough to be assigned positive, that object is never a training target and recall for its class is capped before the first epoch. Small objects fail for a related reason: after enough downsampling they occupy less than one feature-map cell, which is what feature pyramids exist to address. And NMS has its own failure independent of the network: two genuinely overlapping objects produce boxes with high mutual IoU, and the greedy rule deletes one of them.
Backbone forward pass dominates training and inference, so the cost profile is a CNN’s. The heads are cheap. NMS is O(k^2) in the boxes surviving the score threshold, per class, which is negligible at typical counts and becomes real on dense scenes with thousands of survivors. Set-based designs trade that for a Hungarian matching during training, O(n^3) in the number of queries, which is affordable because the query count is fixed and small.
How, where, and why it is used
Time-Series Forecasting
Not applicableThe output is a set of spatial regions with class labels, and the architecture’s prior is spatial locality rather than temporal order; there is no future value anywhere in the formulation.
Anomaly Detection
Not applicableA detector localizes the classes it was trained on, so a defect it has never seen is simply not detected — the opposite of what an anomaly detector must do, which is flag the unfamiliar.
Optimization
AdaptedTwo genuine combinatorial problems sit inside a detector, and neither is gradient descent. Assignment is a matching problem: every prediction must be paired with a ground-truth object or with background, and the set-based designs solve it exactly as a bipartite matching with the Hungarian algorithm rather than by a heuristic IoU rule. Deduplication is a greedy set-selection problem — non-maximum suppression is a hand-written approximation to choosing the highest-scoring non-overlapping subset, which is NP-hard in general.
- Bipartite matching between predictions and ground truth, which is what makes set prediction end-to-end and removes NMS entirely
- Non-maximum suppression as a greedy approximation to maximum-weight independent set on the overlap graph
- Soft-NMS and clustering variants as alternative heuristics for the same underlying selection problem
It is a clear case of a learned system with combinatorial problems embedded in it, and of what happens when one of them is left outside the model: NMS is not differentiable, not trained, and has failure modes the network cannot compensate for because it never sees them. Moving the matching inside training — which is the DETR contribution — is a good demonstration that identifying the combinatorial structure and solving it properly can replace a heuristic that everyone had accepted as inherent.
- Define the matching cost to include both classification and localization, or the assignment optimizes one at the expense of the other
- Use an IoU-based box loss rather than a coordinate loss, so the training objective and the evaluation metric are the same quantity
Compare the greedy suppression against an exact selection on small instances to see what the heuristic costs, and measure recall on crowded scenes specifically — that is where the greedy approximation loses and an aggregate mAP hides it.
- Tuning the NMS threshold on non-crowded validation data and deploying into crowds, where the same threshold deletes real objects
- Treating assignment as a fixed rule rather than a design choice, when it determines what the network is actually supervised on
- Optimizing a coordinate loss while evaluating IoU, so the training objective and the metric disagree about what a good box is
Computer Vision
Primary fitA convolutional backbone extracts features, a feature pyramid supplies them at several scales, and heads predict a class distribution and box offsets at every position and anchor. One-stage designs do this directly; two-stage designs first propose regions and then refine them, trading latency for accuracy. Set-based designs replace the anchors and the suppression with a fixed set of learned queries and a matching loss.
- Autonomous driving and robotics perception, where the deliverable is what is where rather than what is present
- Retail and industrial counting and inspection, where the count and the position are the output
- Surveillance and safety monitoring, usually feeding a tracker that adds temporal identity
- Medical imaging localization, where a bounding region is the clinically useful output
It is the canonical structured-output vision problem, and everything difficult about it comes from the output being a set of variable size rather than from the recognition. The practical shape of the field follows from that: the accuracy-latency trade between one-stage and two-stage designs, the imbalance that motivated focal loss, and the post-processing seam that motivated set prediction. What a detector does not do is anything about identity over time or about objects it was not trained on — the first is a tracker’s job and the second is a limitation no amount of data removes.
- Multi-scale features are effectively mandatory: after enough downsampling a small object occupies less than one cell and cannot be localized at all
- Match anchor scales and aspect ratios to the actual object statistics in the target domain, or recall is capped before training begins
- Augment with scale jitter and mosaic-style composition, which is where most of the practical accuracy on small objects comes from
- Keep the box parameterization relative and log-scaled for width and height, so the regression targets have comparable magnitude across object sizes
mAP averaged over IoU thresholds, reported separately by object scale — an aggregate number hides that small-object performance is usually several times worse. Measure latency at the deployed resolution and batch size, since detectors are almost always latency-constrained in the settings that use them.
- NMS deleting one of two genuinely overlapping objects, which is inherent to the greedy rule rather than a tuning failure
- Anchor design copied from a public benchmark whose object statistics do not match the deployment domain
- Reporting mAP at a single IoU threshold, which rewards loose localization
- Class imbalance handled by resampling rather than by loss weighting, which changes the effective data distribution
What it costs to run
Backbone-dominated: GPU-hours to GPU-days depending on resolution and dataset size, with transfer from a pretrained backbone removing most of it. The heads themselves are cheap, and most of the practical training cost is augmentation and input pipeline rather than the model.
One backbone pass plus cheap heads plus NMS. Real-time is achievable on device for one-stage designs at modest resolution, and latency scales roughly with input area — which makes resolution the primary accuracy-latency dial, more than architecture choice.
Driven by domain shift rather than volume: new object classes, new camera positions, new lighting. Fine-tuning from the deployed weights is standard and much cheaper than retraining, and the anchor design usually survives a domain change while the score thresholds do not.
- Track the score distribution of detections, which shifts before accuracy does when the input domain moves
- Monitor detections per image against the expected range — a sharp change usually means a domain shift rather than a change in the world
- Watch the small-object recall specifically, since it degrades first under resolution or camera changes and is invisible in aggregate mAP
- Sample and review suppressed boxes periodically, because NMS failures on crowded scenes never appear in any aggregate metric
- The score threshold and the NMS threshold are inference-time dials, not learned parameters, and they must be re-tuned for each deployment domain — a threshold tuned on a benchmark rarely transfers
- Box coordinate conventions differ between formats and libraries — corner versus centre, absolute versus normalized, x-y versus y-x — and a mismatch produces plausible-looking boxes in the wrong places
- NMS is per class in most implementations and cross-class in others; the difference is invisible until two classes genuinely overlap
- Letterboxing and resizing must be inverted exactly when mapping boxes back to original coordinates, and an off-by-one in that transform is a common silent error
- Aggregate mAP hides scale-specific failure, so small-object performance has to be monitored as its own number or its degradation goes unnoticed
Pros and cons, in context
Produces localization, not just recognition
What is where, which is the output almost every downstream system actually needs — a tracker, a robot, a counter. A classifier’s answer is not usable for any of them.
One-stage designs run in real time on modest hardware
Which is what makes on-device and embedded deployment realistic, and why the accuracy-latency curve rather than peak accuracy is the axis the field competes on.
Transfers well from a pretrained backbone
A few thousand labelled images is often enough to fine-tune a strong detector, because the backbone already knows what edges and textures are. Decisive when labelling is the bottleneck, which for detection it always is — boxes are far more expensive to annotate than labels.
IoU gives one criterion for assignment, training and evaluation
Unusually coherent: the same quantity supervises, deduplicates and scores. That coherence is why IoU-based losses outperform coordinate losses, and it is worth noticing where it holds.
Non-maximum suppression is outside the model and cannot be trained
A greedy hand-written rule whose failure on overlapping objects the network never sees and cannot compensate for. It is the clearest seam in the design, and removing it is the whole motivation for set-based detectors.
Extreme class imbalance is structural
Background outnumbers objects by orders of magnitude, so a plain cross-entropy collapses to predicting nothing while the loss appears to fall. Focal loss or hard-negative mining is required rather than advisable.
Small objects are substantially harder, and aggregate metrics hide it
After downsampling they occupy less than a feature cell. Feature pyramids help and do not equalize, and a single mAP number will not show that small-object performance is several times worse.
Box annotation is expensive and the class set is closed
Labelling boxes costs far more than labelling images, and an object of an unseen class is silently missed rather than flagged. Open-vocabulary detection exists to address the second and does not remove the first.
Intuitive, idiomatic, optimized
The same algorithm in three languages, each progressing through Kent Beck's three stages. Every optimization names the lever it pulls and the trade-off it accepts — cost profiles are illustrative, not measured benchmarks.
The math, transcribed literally. No dependencies.
"""IoU, box decoding and greedy NMS - transcribed.
The three pieces that turn a grid of predictions into a set of boxes. IoU is
the assignment rule, the evaluation metric and the deduplication criterion all
at once, which is unusually coherent and worth noticing.
"""
def iou(box_a, box_b):
"""Intersection over union for two boxes in (x1, y1, x2, y2) form."""
left = max(box_a[0], box_b[0])
top = max(box_a[1], box_b[1])
right = min(box_a[2], box_b[2])
bottom = min(box_a[3], box_b[3])
# Empty intersection when the boxes do not overlap at all.
width = max(0.0, right - left)
height = max(0.0, bottom - top)
intersection = width * height
area_a = max(0.0, box_a[2] - box_a[0]) * max(0.0, box_a[3] - box_a[1])
area_b = max(0.0, box_b[2] - box_b[0]) * max(0.0, box_b[3] - box_b[1])
union = area_a + area_b - intersection
return intersection / union if union > 0.0 else 0.0
def decode(anchor, offsets):
"""Offsets are corrections to an anchor, never absolute coordinates.
Width and height are predicted in LOG space so that a factor-of-two error
costs the same whether the object is 20 pixels or 200 - which keeps the
regression targets comparable across scales.
"""
anchor_width = anchor[2] - anchor[0]
anchor_height = anchor[3] - anchor[1]
anchor_x = anchor[0] + 0.5 * anchor_width
anchor_y = anchor[1] + 0.5 * anchor_height
import math
centre_x = offsets[0] * anchor_width + anchor_x
centre_y = offsets[1] * anchor_height + anchor_y
width = math.exp(offsets[2]) * anchor_width
height = math.exp(offsets[3]) * anchor_height
return [
centre_x - 0.5 * width,
centre_y - 0.5 * height,
centre_x + 0.5 * width,
centre_y + 0.5 * height,
]
def non_maximum_suppression(boxes, scores, threshold=0.5):
"""Greedy deduplication: keep the best box, delete everything it overlaps.
This is not learned and not differentiable. It is a hand-written rule
applied after the network, which is why two genuinely overlapping objects
lose one of their boxes - a failure the network never sees and cannot
compensate for.
"""
order = sorted(range(len(boxes)), key=lambda i: scores[i], reverse=True)
kept = []
while order:
best = order.pop(0)
kept.append(best)
survivors = []
for candidate in order:
if iou(boxes[best], boxes[candidate]) < threshold:
survivors.append(candidate)
order = survivors
return kept
def assign_targets(anchors, ground_truth, positive=0.5, negative=0.4):
"""Which anchors are responsible for which objects.
The most consequential setting in a detector and the least discussed: if
no anchor overlaps an object by enough to be positive, that object is
never a training target and recall for it is capped before training even
begins.
"""
labels = [-1] * len(anchors) # -1 means ignored
matched = [None] * len(anchors)
for a, anchor in enumerate(anchors):
best_iou = 0.0
best_object = None
for g, box in enumerate(ground_truth):
overlap = iou(anchor, box)
if overlap > best_iou:
best_iou = overlap
best_object = g
if best_iou >= positive:
labels[a] = 1
matched[a] = best_object
elif best_iou < negative:
labels[a] = 0
return labels, matchedO(k^2) for suppression and O(anchors * objects) for assignment, in interpreter loops with a fresh survivor list built per iteration.
How this language actually wants it written.
"""Detection heads - typed boxes, vectorized IoU, class-wise NMS."""
from dataclasses import dataclass
import numpy as np
from numpy.typing import NDArray
Vector = NDArray[np.float64]
Matrix = NDArray[np.float64]
@dataclass(frozen=True)
class Boxes:
"""Corner format (x1, y1, x2, y2), validated once at construction.
Coordinate conventions differ between formats and libraries - corner
versus centre, absolute versus normalized, x-y versus y-x - and a
mismatch produces plausible-looking boxes in the wrong places with nothing
raised. Making the convention a type is the cheapest defence available.
"""
corners: Matrix # (n, 4)
def __post_init__(self) -> None:
if self.corners.ndim != 2 or self.corners.shape[1] != 4:
raise ValueError(f"expected (n, 4) corners, got {self.corners.shape}")
if (self.corners[:, 2] < self.corners[:, 0]).any():
raise ValueError("x2 < x1: boxes are not in corner format, or x-y is transposed")
if (self.corners[:, 3] < self.corners[:, 1]).any():
raise ValueError("y2 < y1: boxes are not in corner format, or x-y is transposed")
@property
def areas(self) -> Vector:
widths = self.corners[:, 2] - self.corners[:, 0]
heights = self.corners[:, 3] - self.corners[:, 1]
return widths * heights
def iou_matrix(self, other: "Boxes") -> Matrix:
"""All pairwise IoUs at once, by broadcasting rather than a loop."""
left = np.maximum(self.corners[:, None, 0], other.corners[None, :, 0])
top = np.maximum(self.corners[:, None, 1], other.corners[None, :, 1])
right = np.minimum(self.corners[:, None, 2], other.corners[None, :, 2])
bottom = np.minimum(self.corners[:, None, 3], other.corners[None, :, 3])
intersection = np.clip(right - left, 0.0, None) * np.clip(bottom - top, 0.0, None)
union = self.areas[:, None] + other.areas[None, :] - intersection
# Degenerate boxes give a zero union; guard rather than divide.
return np.where(union > 0.0, intersection / np.maximum(union, 1e-12), 0.0)
def decode(anchors: Boxes, offsets: Matrix) -> Boxes:
"""Offsets are corrections to an anchor, never absolute coordinates.
Width and height are predicted in LOG space so a factor-of-two error costs
the same at any object size, which keeps regression targets comparable
across scales. The exponential is clipped because an unbounded offset
early in training produces an inf and poisons the whole batch.
"""
widths = anchors.corners[:, 2] - anchors.corners[:, 0]
heights = anchors.corners[:, 3] - anchors.corners[:, 1]
centres_x = anchors.corners[:, 0] + 0.5 * widths
centres_y = anchors.corners[:, 1] + 0.5 * heights
predicted_x = offsets[:, 0] * widths + centres_x
predicted_y = offsets[:, 1] * heights + centres_y
predicted_w = np.exp(np.clip(offsets[:, 2], -4.0, 4.0)) * widths
predicted_h = np.exp(np.clip(offsets[:, 3], -4.0, 4.0)) * heights
return Boxes(
np.stack(
[
predicted_x - 0.5 * predicted_w,
predicted_y - 0.5 * predicted_h,
predicted_x + 0.5 * predicted_w,
predicted_y + 0.5 * predicted_h,
],
axis=1,
)
)
def non_maximum_suppression(boxes: Boxes, scores: Vector, threshold: float = 0.5) -> NDArray:
"""Greedy deduplication within one class.
Not learned and not differentiable: a hand-written rule applied after the
network. Two genuinely overlapping objects lose one of their boxes, and
the network never sees that failure so it cannot compensate for it.
"""
if not 0.0 < threshold < 1.0:
raise ValueError(f"threshold must lie in (0, 1), got {threshold}")
order = np.argsort(scores)[::-1]
kept: list[int] = []
while order.size:
best = order[0]
kept.append(int(best))
if order.size == 1:
break
# One vectorized IoU against every remaining candidate, rather than a
# Python loop rebuilding a survivor list each round.
overlaps = boxes.iou_matrix(Boxes(boxes.corners[order[1:]]))[best]
order = order[1:][overlaps < threshold]
return np.asarray(kept, dtype=np.int64)
def batched_nms(
boxes: Boxes, scores: Vector, labels: NDArray, threshold: float = 0.5
) -> NDArray:
"""Class-wise suppression, which is what almost every implementation
means by NMS.
Cross-class suppression would delete a person standing in front of a car.
The difference is invisible until two classes genuinely overlap, which is
exactly when it matters.
"""
kept: list[NDArray] = []
for label in np.unique(labels):
rows = np.flatnonzero(labels == label)
selected = non_maximum_suppression(Boxes(boxes.corners[rows]), scores[rows], threshold)
kept.append(rows[selected])
return np.concatenate(kept) if kept else np.empty(0, dtype=np.int64)What changed: The box format becomes a validated type rather than a bare array, which is the cheapest available defence against the error that actually happens in detection code: coordinate conventions differ between formats and libraries, and a mismatch produces plausible-looking boxes in the wrong places with nothing raised. Pairwise IoU becomes a broadcast matrix rather than a double loop, and the suppression loop uses it to filter survivors in one vectorized step instead of rebuilding a list. Suppression also becomes explicitly class-wise, since cross-class suppression would delete a person standing in front of a car — a difference invisible until two classes genuinely overlap. The decode clips the exponential, because an unbounded offset early in training produces an inf that poisons the batch.
O(k^2) suppression with each round vectorized, and O(anchors * objects) assignment as one broadcast matrix.
Faster, and the justification for each change.
"""Detection post-processing - preallocated, sorted once, area cached."""
import numpy as np
from numpy.typing import NDArray
Vector = NDArray[np.float64]
Matrix = NDArray[np.float64]
class DetectionPostProcessor:
"""Suppression over a batch, with the per-round allocations removed.
The naive loop rebuilds an index array and recomputes areas on every
iteration. Neither is necessary: the areas are fixed once the boxes exist,
and survivors can be tracked in a preallocated boolean mask instead of a
shrinking array.
Score thresholding first is the change that matters most in practice.
Suppression is O(k^2) in surviving boxes, so discarding low-confidence
candidates before it - typically 99% of them - is a far larger saving than
anything done inside the loop.
"""
def __init__(self, max_boxes: int) -> None:
# Buffers sized once for the worst case, reused across every image.
self._alive = np.empty(max_boxes, dtype=bool)
self._kept = np.empty(max_boxes, dtype=np.int64)
self._max_boxes = max_boxes
def suppress(
self,
corners: Matrix,
scores: Vector,
score_threshold: float = 0.05,
iou_threshold: float = 0.5,
) -> NDArray:
# Cheap filter before the quadratic step: this typically removes 99%
# of candidates and is worth more than any inner-loop optimization.
surviving = np.flatnonzero(scores > score_threshold)
if surviving.size == 0:
return np.empty(0, dtype=np.int64)
boxes = np.ascontiguousarray(corners[surviving], dtype=np.float64)
order = surviving[np.argsort(scores[surviving])[::-1]]
boxes = np.ascontiguousarray(corners[order], dtype=np.float64)
# Areas computed once for the whole set rather than per comparison.
widths = boxes[:, 2] - boxes[:, 0]
heights = boxes[:, 3] - boxes[:, 1]
areas = widths * heights
n = order.size
alive = self._alive[:n]
alive.fill(True)
count = 0
for index in range(n):
if not alive[index]:
continue
self._kept[count] = order[index]
count += 1
if index + 1 >= n:
break
# One vectorized IoU against every remaining live candidate,
# written through in place so no per-round temporary is created.
rest = slice(index + 1, n)
left = np.maximum(boxes[index, 0], boxes[rest, 0])
top = np.maximum(boxes[index, 1], boxes[rest, 1])
right = np.minimum(boxes[index, 2], boxes[rest, 2])
bottom = np.minimum(boxes[index, 3], boxes[rest, 3])
overlap = np.clip(right - left, 0.0, None)
overlap *= np.clip(bottom - top, 0.0, None)
union = areas[index] + areas[rest] - overlap
np.divide(overlap, np.maximum(union, 1e-12), out=overlap)
# Suppression is a mask update, not an array rebuild.
alive[rest] &= overlap < iou_threshold
return self._kept[:count].copy()
def suppress_by_class(
self,
corners: Matrix,
scores: Vector,
labels: NDArray,
score_threshold: float = 0.05,
iou_threshold: float = 0.5,
) -> NDArray:
"""Class-wise suppression by coordinate offsetting.
Rather than looping over classes, shift each class's boxes into a
disjoint region of the coordinate plane. Boxes of different classes
then have zero overlap by construction, so ONE suppression pass
produces the same result as one pass per class.
"""
if corners.size == 0:
return np.empty(0, dtype=np.int64)
span = float(corners.max()) - float(corners.min()) + 1.0
shifted = corners + (labels.astype(np.float64) * span)[:, None]
return self.suppress(shifted, scores, score_threshold, iou_threshold)What changed: The largest saving is not inside the loop: suppression is quadratic in surviving boxes, so applying the score threshold first — which typically discards 99% of candidates — dominates anything done to the inner step. Beyond that, areas are computed once rather than per comparison, survivors are tracked in a preallocated boolean mask rather than by rebuilding an index array each round, and the IoU is written through in place. Class-wise suppression stops being a loop over classes and becomes a coordinate offset that pushes each class into a disjoint region, so one pass produces the same result — the trick every production implementation uses.
Pre-allocate output arrays and use in-place operations
The alive mask and the kept-index buffer are sized once for the worst case and reused across images, and the IoU is computed through in-place operations rather than allocating a temporary per round.
Trade-off: Fixes a maximum box count at construction and makes the object non-reentrant, so two threads post-processing through one instance would corrupt each other.
Vectorize to NumPy/BLAS (@, np.dot, einsum)
Each round compares the current best against every remaining candidate in one vectorized pass instead of a Python loop over candidates.
Trade-off: The outer loop remains sequential and cannot be vectorized — suppression is inherently order-dependent — so this improves the constant, not the O(k^2).
Ensure contiguous memory layout and a single dtype
Boxes are gathered into one contiguous float64 block in score order, so every slice in the loop is a sequential read rather than a strided gather.
Trade-off: Costs a copy of the surviving boxes up front, which is only worthwhile because the score threshold has already reduced them to a small fraction.
O(k^2) in boxes surviving the score threshold, with the threshold doing most of the work. Illustrative, not a measured benchmark.