Object detection finds every instance of a set of object classes in an image and returns, for each, a bounding box, a class label, and a confidence score. It powers autonomous driving perception, retail shelf analytics, security video, document layout analysis, and industrial inspection.
Parent topic: Computer Vision. See also YOLO.
A box is typically (xmin, ymin, xmax, ymax) or (cx, cy, w, h). Intersection over Union is the core overlap measure used for matching predictions to ground truth, for suppressing duplicates, and in losses such as GIoU/CIoU that directly optimise overlap.
Dense detectors emit many overlapping boxes per object. NMS greedily keeps the highest-scoring box and removes neighbours above an IoU threshold. Too low a threshold merges adjacent objects in crowds; too high leaves duplicates. Soft-NMS decays scores instead of deleting.
Torchvision ships a pretrained Faster R-CNN; replacing its box predictor adapts it to custom classes.
import torch
from torchvision.io import read_image
from torchvision.models.detection import fasterrcnn_resnet50_fpn_v2, FasterRCNN_ResNet50_FPN_V2_Weights
from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
weights = FasterRCNN_ResNet50_FPN_V2_Weights.DEFAULT
model = fasterrcnn_resnet50_fpn_v2(weights=weights, box_score_thresh=0.5).eval()
img = read_image("street.jpg")
with torch.no_grad():
out = model([weights.transforms()(img)])[0]
names = weights.meta["categories"]
for box, label, score in zip(out["boxes"], out["labels"], out["scores"]):
print(names[label], round(score.item(), 2), [round(v) for v in box.tolist()])
# --- adapt to 3 custom classes (+1 background) ---
num_classes = 4
in_feat = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(in_feat, num_classes)
model.train()
# each target: {"boxes": FloatTensor[N,4] (xyxy, pixels), "labels": Int64Tensor[N]}
images = [torch.rand(3, 480, 640)]
targets = [{"boxes": torch.tensor([[50., 60., 200., 220.]]), "labels": torch.tensor([1])}]
losses = model(images, targets) # dict of classifier/box/objectness/rpn losses
sum(losses.values()).backward()For real-time needs the ultralytics YOLO package wraps training, export (ONNX, TensorRT) and inference in a few lines; check its licence (AGPL) for commercial use.
For each class, predictions are sorted by confidence and matched to ground truth at an IoU threshold, giving a precision-recall curve. AP is the area under it; mAP averages AP over classes. COCO reports mAP@[.50:.95] (averaged over ten IoU thresholds) plus breakdowns by object size. For production, also track recall at your operating threshold, false positives per image, and latency on the target hardware.
Take a ground-truth box (10, 10, 50, 50) and a prediction (30, 30, 70, 70) in (xmin, ymin, xmax, ymax) form. Each box covers 40 × 40 = 1,600 pixels. The intersection is (30, 30, 50, 50), or 20 × 20 = 400. The union is 1,600 + 1,600 − 400 = 2,800. IoU = 400 / 2,800 ≈ 0.14, well below the usual 0.5 match threshold, so this prediction is a false positive and the object counts as a miss. Shifting the prediction to (15, 15, 55, 55) raises the intersection to 35 × 35 = 1,225 and IoU to 1,225 / 1,975 ≈ 0.62, now a true positive at IoU 0.5 but not at 0.75. COCO-style mAP averages over thresholds from 0.50 to 0.95 precisely to reward tight localization.
During training the same overlap logic assigns targets. Anchor-based detectors label an anchor positive if its IoU with a ground-truth box exceeds a threshold (often 0.5 to 0.7) and negative below a lower threshold. Modern assigners such as ATSS, SimOTA and task-aligned assignment choose positives adaptively per object, which removes several hand-tuned thresholds. DETR-style models skip assignment heuristics entirely and use Hungarian matching to pair each ground-truth box with exactly one prediction.
Almost every current detector has the same three parts. The backbone, a ResNet-style CNN or a Vision Transformer pretrained on image classification, extracts feature maps at several resolutions. The neck, usually a feature pyramid network or one of its variants, mixes semantic information from deep layers into shallow, high-resolution layers. The head predicts, at each location on each scale, class probabilities and box regression values. The loss combines a classification term (cross-entropy or focal loss) with a box term (L1 plus an IoU-based loss such as GIoU or CIoU).
| Need | Reasonable starting point |
|---|---|
| Real-time on edge devices or video | A small YOLO variant or RT-DETR, exported to ONNX or TensorRT |
| Highest accuracy, latency less critical | Two-stage (Cascade R-CNN) or a large DETR-family model |
| Objects plus pixel masks | Mask R-CNN or a segmentation model; see Image Segmentation |
| New classes without labeled data | Open-vocabulary detector (Grounding DINO, OWL-ViT), then fine-tune if needed |
| Rotated objects (aerial, documents) | Oriented-box detectors that predict an angle |
Deployed detectors usually run inside a larger pipeline: video decoding, batching, the detector, a tracker (SORT, ByteTrack) that links boxes across frames into object identities, and business logic such as counting line crossings. Measure end-to-end latency, not just model time, because pre- and post-processing can dominate. Monitor the score distribution and detection counts per camera over time; a sudden change often means a moved camera, new lighting or a dirty lens rather than a model bug. Typical uses include pedestrian and vehicle detection in driving, defect detection on production lines, shelf-gap detection in retail, wildlife counting in camera traps, table and figure detection in document AI, and lesion detection in medical imaging.
Further reading. Girshick (2015), "Fast R-CNN"; Ren et al. (2015), "Faster R-CNN", arXiv:1506.01497; Redmon et al. (2016), "You Only Look Once"; Lin et al. (2017), "Feature Pyramid Networks for Object Detection" and "Focal Loss for Dense Object Detection"; Carion et al. (2020), "End-to-End Object Detection with Transformers", arXiv:2005.12872; Lin et al. (2014), "Microsoft COCO: Common Objects in Context".