Facial recognition identifies or verifies a person from an image of their face. Modern systems do not classify identities directly; they map each face to an embedding vector such that faces of the same person are close and different people are far apart, then compare embeddings. That makes it possible to enrol new people without retraining.
It is also one of the most regulated and ethically sensitive uses of computer vision. Parent topic: Computer Vision.
Detection finds faces and five landmarks (eyes, nose, mouth corners) — see Object Detection. Alignment applies a similarity transform so the landmarks land at canonical positions in a 112×112 crop; this step matters as much as the model. The embedding network (typically a ResNet-50/100 or a ViT) outputs a normalised vector, and matching is cosine similarity against stored templates.
Training uses a classification head over thousands of identities, but with a margin-based softmax. ArcFace adds an angular margin m to the angle between an embedding and its class centre: logit = s·cos(θy + m). After training the head is discarded and only the embedding is kept. Earlier FaceNet used triplet loss on (anchor, positive, negative) triples, which needs careful hard-negative mining.
Raising the threshold lowers false accepts and raises false rejects. Report true accept rate at a fixed false accept rate on a held-out set that reflects deployment (camera, lighting, demographics). For 1:N search, false positive identification rate grows with gallery size, so thresholds must be stricter for large galleries.
The open-source insightface package bundles a detector, alignment, and an ArcFace model (check its model licence before commercial use).
import cv2, numpy as np
from insightface.app import FaceAnalysis
app = FaceAnalysis(name="buffalo_l", providers=["CPUExecutionProvider"])
app.prepare(ctx_id=-1, det_size=(640, 640))
def embed(path):
faces = app.get(cv2.imread(path))
if len(faces) != 1:
raise ValueError(f"{path}: expected 1 face, found {len(faces)}")
return faces[0].normed_embedding # 512-d, unit length
enrolled = embed("alice_id_photo.jpg")
probe = embed("camera_frame.jpg")
sim = float(np.dot(enrolled, probe)) # cosine similarity
THRESH = 0.35 # calibrate on your own data
print(f"similarity={sim:.3f}", "MATCH" if sim >= THRESH else "NO MATCH")For 1:N search, store normalised embeddings in a vector index (FAISS inner-product index) and return the top hit only if it clears the threshold.
The same embedding model serves two very different tasks. Verification (1:1) compares a probe with one enrolled template — unlocking a phone, matching a selfie to an ID document. Identification (1:N) searches a gallery for the best match — finding a person in a set of enrolled employees, or de-duplicating accounts. Identification is strictly harder: every extra gallery entry is another chance for a false match, so a threshold that gives a tiny false accept rate in 1:1 can yield a noticeable false positive identification rate against a gallery of a million.
A useful approximation: if each independent comparison has false match probability p, the chance that at least one of N non-mated templates exceeds the threshold is 1 − (1 − p)N, roughly N·p when N·p is small. With p = 10−6 and N = 100,000 that is already around 10%. Real comparisons are not independent, so measure on your own gallery, but the direction is reliable: bigger gallery, stricter threshold.
Suppose you collect 2,000 genuine pairs (same person, different photos) and 200,000 impostor pairs from a validation set that matches your cameras. You sweep the cosine threshold and record:
| Threshold | False accept rate (impostors above) | False reject rate (genuine below) |
|---|---|---|
| 0.20 | high | very low |
| 0.35 | moderate | low |
| 0.50 | very low | noticeably higher |
The values are illustrative of the shape, not of any particular model; the actual cosine range depends on the model and preprocessing. The procedure is what matters: fix the false accept rate the use case can tolerate (for example 1 in 10,000 for a low-risk convenience feature, far lower for access control), read off the threshold from the impostor distribution, then report the false reject rate at that point. Plotting the full trade-off as a DET curve (log-scale FAR vs FRR) shows how much headroom there is. With only 200,000 impostor pairs you cannot reliably measure a FAR below about 1 in 100,000; the estimate needs many more impostor comparisons than the target rate's inverse.
Repeat the sweep per demographic group and per capture condition. A single global threshold can produce very different error rates across groups, which is the practical meaning of the bias findings in NIST's evaluations.
| Loss | Idea | Trade-off |
|---|---|---|
| Softmax (plain) | Classify training identities | Separable but not discriminative enough for open-set matching |
| Contrastive / triplet (FaceNet, 2015) | Pull anchor–positive together, push anchor–negative apart by a margin | Needs hard-negative mining; slow to converge on large datasets |
| SphereFace (2017) | Multiplicative angular margin | Effective but harder to train stably |
| CosFace (2018) | Additive cosine margin: s·(cos θ − m) | Simple, stable |
| ArcFace (2019) | Additive angular margin: s·cos(θ + m) | Widely used default; clear geometric interpretation |
All margin losses do the same thing geometrically: they force embeddings of one identity into a tight cone on the unit hypersphere, leaving angular gaps between identities. That is why cosine similarity is the natural matching score. Typical settings in the published work use a scale s of about 64 and a margin around 0.5 radians for ArcFace, but these interact with dataset size and noise level, so treat them as starting points.
Legitimate low-risk uses include on-device phone unlock, opt-in photo grouping in personal libraries, and consent-based identity verification during account onboarding. Many organisations decide the regulatory and reputational risk of open-ended surveillance use is not worth it; that decision belongs in the design review, not after deployment.