Anomaly detection is about finding the rare points that do not belong, usually with few or no labels. This page surveys the main method families, explains Isolation Forest and LOF, compares detectors in code, and covers what it takes to run detection in production.
Anomaly detection (outlier or novelty detection) finds observations that do not fit the pattern of the rest: fraudulent transactions, failing machines, intrusions, data-quality bugs. Labels are usually scarce and anomalies are rare and varied, so most methods are unsupervised or semi-supervised (trained on data assumed to be normal) and output a score rather than a class. Two settings are worth separating: outlier detection (training data is contaminated, find the odd ones) and novelty detection (training data is clean, flag new points that differ).
| Family | Idea | Examples |
|---|---|---|
| Statistical | Low probability under a fitted distribution | z-score, robust MAD, Gaussian / GMM likelihood, Mahalanobis distance |
| Distance / density | Far from neighbors or in sparse regions | k-NN distance, Local Outlier Factor (LOF), DBSCAN noise |
| Isolation | Easy to separate with random splits | Isolation Forest, Extended IF |
| Boundary | Learn a tight region around normal data | One-Class SVM, Deep SVDD |
| Reconstruction | Large error when compressed and rebuilt | PCA residual, autoencoders |
| Forecast residual | Large deviation from a prediction | Seasonal decomposition, ARIMA / Prophet residuals |
LOF compares a point's local density to that of its neighbors: a score near 1 is normal, well above 1 means the point is sparser than its neighborhood. That makes it work when clusters have different densities, where a global distance threshold fails.
Isolation Forest (Liu, Ting and Zhou 2008) builds many random trees, each splitting on a random feature at a random value between its min and max. Anomalies are few and different, so they end up isolated in a leaf after few splits. The score is based on the average path length E[h(x)] normalized by the expected path length of an unsuccessful binary-search-tree lookup, c(n): s(x) = 2-E[h(x)] / c(n). Scores near 1 are anomalous, well below 0.5 normal. It trains on small subsamples (256 by default), is linear in data size, needs no distance computations, and is the usual first choice on tabular data.
import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor
from sklearn.svm import OneClassSVM
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import roc_auc_score, average_precision_score
rng = np.random.default_rng(0)
normal = rng.normal(0, 1, size=(2000, 6))
anom = rng.uniform(-6, 6, size=(40, 6))
X = np.vstack([normal, anom]); y = np.r_[np.zeros(2000), np.ones(40)]
X = StandardScaler().fit_transform(X)
models = {
"iforest": IsolationForest(n_estimators=300, contamination="auto", random_state=0),
"lof": LocalOutlierFactor(n_neighbors=35, novelty=False),
"ocsvm": OneClassSVM(nu=0.02, gamma="scale"),
}
for name, m in models.items():
if name == "lof":
m.fit_predict(X); score = -m.negative_outlier_factor_
else:
m.fit(X); score = -m.score_samples(X) # higher = more anomalous
print(f"{name:8s} ROC-AUC={roc_auc_score(y, score):.3f} "
f"PR-AUC={average_precision_score(y, score):.3f}")
# Turn scores into alerts by budget, not by a magic contamination value
threshold = np.quantile(score, 0.99) # top 1% -> review queueEvaluate with precision-recall AUC or precision at the top-k alerts, not accuracy: with 2% anomalies, predicting "all normal" is 98% accurate and useless.
contamination to a guessed rate and treating the resulting labels as truth.Start simple, because a well-chosen statistic is often good enough and always easy to explain. Take eight readings: 10, 11, 12, 11, 10, 12, 11 and 50. The mean is 15.9 and the standard deviation 12.9, so the z-score of 50 is only (50 − 15.9) / 12.9 ≈ 2.6, below the common cut-off of 3. The outlier inflated both the mean and the spread it is measured against, an effect called masking. The robust alternative uses the median (11) and the median absolute deviation (MAD, here 1). The modified z-score 0.6745 × (50 − 11) / 1 ≈ 26 flags the point unmistakably; Iglewicz and Hoaglin suggest treating modified z-scores above 3.5 as potential outliers.
For several correlated features, the multivariate analogue is the Mahalanobis distance √((x − μ)T Σ−1 (x − μ)), which measures distance in units of the data's own covariance, so a point that is ordinary on each feature separately but has an unusual combination (tall and very light) still stands out. Estimate μ and Σ robustly, for example with the minimum covariance determinant (sklearn.covariance.EllipticEnvelope), or the anomalies will again mask themselves.
LOF (Breunig et al., 2000) defines, for each point, the k-distance to its k-th nearest neighbour, and a reachability distance from p to o as max(k-distance(o), d(p, o)), which smooths out tiny distances inside dense groups. The local reachability density of p is the inverse of its mean reachability distance to its k neighbours. Finally, LOF(p) is the average ratio of its neighbours' densities to its own. A point as dense as its neighbours scores about 1; a point five times sparser than its neighbourhood scores about 5. Typical n_neighbors is 20; it should exceed the size of any small group of anomalies you want detected, or such a group becomes its own normal neighbourhood. In scikit-learn, use novelty=True to score unseen data.
Isolation Forest scores can be read the same concrete way. With the default subsample of 256, the normalising constant c(256) ≈ 10.2. A point isolated after 4 splits on average scores 2−4/10.2 ≈ 0.76; one needing 10 splits scores about 0.51; one needing 12 scores about 0.44 and looks normal.
| Data | Reasonable first choice | Notes |
|---|---|---|
| One metric, roughly stable | Robust z-score (median / MAD) | Explainable; add seasonal baselines for time-of-day effects |
| Tabular, mixed features | Isolation Forest | Fast, few assumptions; encode categoricals sensibly |
| Clusters of different density | LOF | Scale features; quadratic-ish cost on large data without indexes |
| Images, audio, logs as sequences | Autoencoder or pretrained-embedding + k-NN distance | Train on normal data only for novelty detection |
| Metrics over time | Forecast residuals (seasonal model) with robust thresholds | Handle holidays and deploys as known events |
| Some labeled incidents | Supervised model plus an unsupervised detector | Supervised catches known patterns, unsupervised the new ones |
Ensembles of detectors, combined after converting each score to a rank or percentile, are often more robust than any single method, because different detectors fail on different anomaly types. The PyOD library provides consistent interfaces for many detectors and makes this easy to try.
When labels exist, report precision-recall AUC and precision at the alert budget, and look at recall per anomaly type rather than overall. For time series, decide whether you score points or events: an incident spanning 30 minutes is one event, and detecting it at minute 2 is very different from minute 28, so report detection delay. Be cautious with "point-adjusted" scoring used in some time-series benchmarks, which counts a whole anomalous segment as detected if any single point is flagged and can make even random scores look strong.
When labels do not exist, inject synthetic anomalies with known properties (spikes, level shifts, swapped fields) to sanity-check recall, and review a sample of the top-scoring events with domain experts. Operationally, group related alerts, suppress repeats, and route each alert with the features that drove it. Track alert volume, reviewer precision and time-to-acknowledge as first-class metrics; a detector that is accurate but ignored because it is noisy has no value. Typical deployments include payment fraud, account takeover, network intrusion, manufacturing quality, predictive maintenance from vibration and temperature sensors, and data-pipeline monitoring (row counts, null rates, schema changes). See time-series forecasting for the residual-based approach.
Further reading: Chandola, Banerjee & Kumar, "Anomaly Detection: A Survey" (ACM Computing Surveys, 2009); Liu, Ting & Zhou, "Isolation Forest" (ICDM 2008); Breunig, Kriegel, Ng & Sander, "LOF: Identifying Density-Based Local Outliers" (SIGMOD 2000); Iglewicz & Hoaglin, How to Detect and Handle Outliers (1993); Ruff et al., "Deep One-Class Classification" (ICML 2018); Zhao, Nasrullah & Li, "PyOD" (JMLR, 2019).