Gradient-boosted decision trees are the default winning model on structured, tabular data. This page explains the algorithm from residual fitting to the second-order gain formula used by XGBoost and LightGBM, compares the major libraries, and gives a production-style training recipe with early stopping.
Gradient boosting (Friedman 2001) builds an additive model one small tree at a time. Each new tree is fit to the negative gradient of the loss with respect to the current predictions — for squared error that is simply the residual y - F(x). The tree's output is scaled by a learning rate η (typically 0.01–0.1) and added to the running model. Where bagging averages independent deep trees to cut variance, boosting chains shallow trees to cut bias.
Because it works on gradients, the same algorithm handles any differentiable loss: log-loss for classification, Huber or quantile loss for robust regression, Poisson or Tweedie for counts and insurance claims, and pairwise/listwise losses for ranking.
Modern libraries (XGBoost, LightGBM, CatBoost) use a second-order Taylor expansion of the loss. For each row i they compute gradient gi and Hessian hi of the loss at the current prediction. For a candidate tree, the optimal value of leaf j with row set Ij is:
wj* = - Σgi / (Σhi + λ)
and the gain of splitting a node into left/right children is:
Gain = ½ [ GL2/(HL+λ) + GR2/(HR+λ) - (GL+GR)2/(HL+HR+λ) ] - γ
λ is L2 regularization on leaf weights, γ a minimum gain to split. Split finding only needs sums of g and h per bin, which is why histogram binning (bucketing each feature into ~256 bins) makes training so fast: it turns sorting into counting.
gi, hi for all rows at current predictions.subsample) and columns (colsample_bytree).wj*, multiply by η, add to predictions.k rounds.XGBoost popularized the regularized second-order objective and grows trees level-wise by default (it also supports grow_policy="lossguide"). LightGBM grows leaf-wise — always splitting the leaf with the largest gain — which reaches lower loss with fewer leaves but can overfit small data unless num_leaves and min_child_samples are constrained; it also adds gradient-based one-side sampling and exclusive feature bundling for speed. CatBoost uses symmetric (oblivious) trees and ordered target statistics for categorical features, which reduces target leakage and usually needs the least tuning on categorical-heavy data. scikit-learn's HistGradientBoostingClassifier is a LightGBM-style implementation with no extra dependency.
| XGBoost | LightGBM | CatBoost | |
|---|---|---|---|
| Default growth | Level-wise | Leaf-wise | Symmetric |
| Categoricals | Native (enable_categorical) | Native (categorical_feature) | Native, ordered encoding |
| Main complexity knob | max_depth | num_leaves | depth |
| GPU | Yes | Yes | Yes |
Learning rate and number of rounds trade off against each other: halving η roughly doubles the rounds needed and usually improves generalization slightly. The standard recipe is to fix a small learning rate (0.03–0.1), set the round count high, and let early stopping on a validation set pick the iteration. Unlike a random forest, boosting does overfit as rounds increase, so early stopping is not optional.
import lightgbm as lgb
import numpy as np
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error
X, y = fetch_california_housing(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=0)
X_tr, X_va, y_tr, y_va = train_test_split(X_tr, y_tr, test_size=0.2, random_state=0)
model = lgb.LGBMRegressor(
n_estimators=5000, # upper bound; early stopping picks the real number
learning_rate=0.05,
num_leaves=31, # main capacity knob (leaf-wise growth)
min_child_samples=20,
subsample=0.8, subsample_freq=1,
colsample_bytree=0.8,
reg_lambda=1.0,
)
model.fit(X_tr, y_tr,
eval_set=[(X_va, y_va)],
callbacks=[lgb.early_stopping(100), lgb.log_evaluation(200)])
print("best iteration:", model.best_iteration_)
print("test MAE: %.3f" % mean_absolute_error(y_te, model.predict(X_te)))
# The equivalent in XGBoost:
# xgb.XGBRegressor(n_estimators=5000, learning_rate=0.05, max_depth=6,
# subsample=0.8, colsample_bytree=0.8,
# early_stopping_rounds=100, tree_method="hist")Keep three splits: train, validation (for early stopping), and test (untouched). Using the test set for early stopping leaks it into model selection.
num_leaves / max_depth and min_child_samples / min_child_weight.subsample and colsample_bytree in 0.6–1.0.reg_lambda, reg_alpha, min_split_gain.Bayesian search (Optuna) over 50–200 trials with early stopping inside each trial is the usual practice. Monotonic constraints (monotone_constraints) are worth knowing for regulated domains where a feature must only push predictions one way.
num_leaves on a few thousand rows memorizes. Lower it or switch to depth limits.With squared loss ½(y − F)2, the gradient is g = F − y and the Hessian is 1, so the leaf formula w* = −Σg / (Σh + λ) with λ = 0 is simply the mean residual in the leaf. Take four rows, x = 1, 2, 3, 4 and y = 1, 2, 6, 7, and a learning rate of 0.5.
F0 = 4. Residuals are −3, −2, 2, 3; MSE is 6.5.x ≤ 2, with leaf values −2.5 and +2.5. Adding half of that gives predictions 2.75 and 5.25, residuals −1.75, −0.75, 0.75, 1.75 and MSE about 1.81.Two things are visible. Training error falls geometrically, and with a learning rate of 1.0 it would have dropped to the within-group error in a single round — shrinkage deliberately slows this so later trees, built on different splits and row samples, share the work. And a stump can never separate rows 1 and 2; a deeper tree (or later rounds that pick x ≤ 1) is needed, which is why tree depth controls the order of feature interactions the ensemble can represent.
| Problem | LightGBM objective | Notes |
|---|---|---|
| Regression, mean | regression | Squared error; sensitive to outliers. |
| Regression with outliers | huber, regression_l1 | Bounded gradients for large residuals. |
| Prediction intervals | quantile with alpha | Train one model per quantile, e.g. 0.1 and 0.9. |
| Counts | poisson | Log link; predictions stay positive. |
| Claims, zero-inflated amounts | tweedie | Common in insurance pricing. |
| Binary classification | binary | Log-loss; check calibration before using as probabilities. |
| Multiclass | multiclass | One tree per class per round; cost scales with class count. |
| Ranking | lambdarank | Needs query groups; optimizes an NDCG-oriented objective. |
Choosing the loss is a modeling decision, not a tuning detail: it decides which statistic of the conditional distribution (mean, median, quantile) the model estimates. Match it to the business metric, and evaluate with that metric rather than whatever the library reports by default.
On medium-sized tabular data, boosted trees remain a strong default compared with deep networks; benchmarks such as Grinsztajn et al. (2022) found tree ensembles ahead on many such datasets. Compare against a random forest and a regularized linear model before committing.