Computer Vision

Computer vision (CV) is the field of extracting structured information from images and video: what is in the scene, where it is, and which pixels belong to it. Modern CV is built on deep learning — Convolutional Neural Networks and Vision Transformers — pretrained on large datasets and adapted to specific tasks.

This landing page maps the main tasks and links to a deep dive for each.

1. Topics

Image Classification

Assign one label per image with CNNs or ViTs; transfer learning, augmentation, and evaluation metrics.

Object Detection

Locate and classify objects with bounding boxes: two-stage R-CNN, single-stage YOLO, DETR, IoU and mAP.

Image Segmentation

Per-pixel labels: semantic, instance, and panoptic segmentation with U-Net, Mask R-CNN, and SAM.

Facial Recognition

Detect, align, embed, and match faces with metric learning (ArcFace); accuracy, bias, and privacy concerns.

Document AI & OCR

Extract text, tables, and key-value pairs from scanned documents and forms with Azure AI Document Intelligence.

Medical Imaging

CV for X-ray, CT, MRI, and pathology: DICOM, 3D U-Net, small-data strategies, and regulatory validation.

Vision-Language (CLIP)

Contrastive image-text embeddings for zero-shot classification, image search, and multimodal LLMs.

Vision Services (Azure AI)

Managed vision APIs for captioning, tagging, OCR, and spatial analysis without training your own models.

2. Core Tasks

Computer Vision Tasks — Increasing Output Detail Classification one label / image "cat" Detection boxes + labels [x,y,w,h], cls Semantic Seg. label per pixel H×W class map Instance Seg. mask per object N masks + cls More detail = more labeling cost: image labels are cheap; pixel masks can cost minutes per image.

The tasks form a ladder of output granularity. Classification answers "what"; detection adds "where" as boxes; semantic segmentation labels every pixel by class; instance segmentation separates individual objects of the same class; panoptic segmentation combines both. Specialised tasks — face recognition, OCR, pose estimation, medical image analysis — build on these primitives.

3. The Standard Pipeline

Typical Vision Pipeline Images capture, label Preprocess resize, normalize Backbone CNN / ViT features Task Head cls / box / mask Deploy ONNX, TensorRT monitor drift, mine hard examples, relabel, retrain Backbone reuse: one pretrained feature extractor serves many heads — transfer learning is the default.

Nearly every production CV system shares this shape: a pretrained backbone (ResNet, EfficientNet, ConvNeXt, ViT, or a CLIP/DINOv2 encoder) extracts features, and a lightweight task-specific head produces the output. Most effort goes into data — label quality, class balance, and coverage of real deployment conditions (lighting, camera, angle) — rather than architecture.

Choose managed services such as Azure AI Vision when generic tags, captions, or OCR suffice; train or fine-tune your own model when the domain is specialised or latency, cost, and data residency require it.