Four libraries cover most practical machine learning in Python: scikit-learn for classical ML on tabular data, PyTorch as the dominant deep learning framework in research and increasingly in production, Keras / TensorFlow for high-level model building and TensorFlow's deployment tooling, and Hugging Face as the hub and library layer for pretrained transformer models. They are not really competitors — they sit at different layers and are routinely used together. This page maps where each fits, shows the same task in each, and links to the dedicated deep-dive pages.
At the bottom, a tensor library provides n-dimensional arrays on accelerators plus automatic differentiation: PyTorch, TensorFlow, and JAX. Above that, high-level APIs remove boilerplate: Keras (which since Keras 3 runs on TensorFlow, JAX, or PyTorch backends), PyTorch Lightning, and Hugging Face Accelerate. At the top, model hubs ship pretrained weights and task pipelines. scikit-learn lives beside this stack: it is built on NumPy and SciPy, runs on CPU, and focuses on classical algorithms with a uniform fit/predict API.
Dynamic computation graphs, nn.Module, autograd, and explicit training loops; the default for research and custom models.
Sequential and functional APIs, compile/fit, callbacks, and TensorFlow's serving and mobile tooling.
Transformers pipeline, AutoModel/AutoTokenizer, the Hub, Datasets, and fine-tuning with Trainer and PEFT.
Estimators, transformers, Pipeline, cross-validation and grid search for classical ML on tabular data.
PyTorch is define-by-run: the computation graph is built as Python executes, so you debug with print statements and a normal debugger, and control flow is ordinary Python. You write the training loop yourself, which is verbose but makes custom losses, multiple optimizers, and unusual training schemes straightforward. torch.compile (PyTorch 2.x) captures and optimizes graphs for speed without changing that model.
Keras is declarative: define layers, compile with an optimizer and loss, call fit. It is the quickest way from idea to a trained standard model, and customizing train_step covers most advanced needs. TensorFlow traces functions into graphs with tf.function and brings mature deployment tooling (TF Serving, TFLite, TF.js). JAX offers NumPy-style functional transformations (grad, jit, vmap, pmap) compiled with XLA and is popular for large-scale research on TPUs.
scikit-learn standardizes everything as estimators: fit(X, y), predict(X), transform(X), composed with Pipeline and tuned with GridSearchCV. Hugging Face Transformers wraps PyTorch (and historically TF/JAX) models behind AutoModel classes and a pipeline() one-liner.
Iris classification, four ways. The point is the shape of the code, not the accuracy.
# ---- scikit-learn: classical ML, a few lines ----
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
clf = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_tr, y_tr)
print("sklearn acc:", clf.score(X_te, y_te))
# ---- PyTorch: explicit model and training loop ----
import torch, torch.nn as nn
scaler = StandardScaler().fit(X_tr)
Xt = torch.tensor(scaler.transform(X_tr), dtype=torch.float32); yt = torch.tensor(y_tr)
Xv = torch.tensor(scaler.transform(X_te), dtype=torch.float32); yv = torch.tensor(y_te)
model = nn.Sequential(nn.Linear(4, 32), nn.ReLU(), nn.Linear(32, 3))
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
loss_fn = nn.CrossEntropyLoss()
for epoch in range(200):
model.train()
loss = loss_fn(model(Xt), yt)
opt.zero_grad(); loss.backward(); opt.step()
model.eval()
with torch.no_grad():
print("torch acc:", (model(Xv).argmax(1) == yv).float().mean().item())
# ---- Keras 3: declarative compile/fit ----
import keras
km = keras.Sequential([keras.Input((4,)), keras.layers.Dense(32, activation="relu"),
keras.layers.Dense(3, activation="softmax")])
km.compile(optimizer=keras.optimizers.Adam(1e-2), loss="sparse_categorical_crossentropy", metrics=["accuracy"])
km.fit(scaler.transform(X_tr), y_tr, epochs=100, batch_size=16, verbose=0)
print("keras acc:", km.evaluate(scaler.transform(X_te), y_te, verbose=0)[1])
# ---- Hugging Face: pretrained model, zero training (text task) ----
from transformers import pipeline
clf_txt = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
print(clf_txt("The petals are long and the flower is deep violet.",
candidate_labels=["setosa", "versicolor", "virginica"]))
The last block is deliberately different: Hugging Face's value is starting from a pretrained model on unstructured data, not training a tiny tabular model. For tabular problems, scikit-learn plus gradient boosting is usually the right tool.
| scikit-learn | PyTorch | Keras / TF | Hugging Face | |
|---|---|---|---|---|
| Primary use | Classical ML, tabular | Custom deep learning | Fast DL prototyping, TF deployment | Pretrained NLP / vision / audio |
| GPU | No (some via cuML) | Yes | Yes (and TPU) | Via PyTorch |
| Training loop | fit | Write your own (or Lightning) | fit | Trainer / Accelerate |
| Debugging | Simple | Python-native, eager | Eager by default; graph mode can obscure errors | Inherits from PyTorch |
| Deployment | joblib / ONNX | TorchScript, ONNX, torch.export | SavedModel, TF Serving, TFLite | Inference endpoints, TGI, vLLM |
For LLM serving specifically, see vLLM and quantization.
model.train()/model.eval()) changes dropout and batch-norm behavior.Pipeline enforces this inside cross-validation.safetensors for model weights.Training and serving often use different software. A model trained in PyTorch can be exported with torch.export or to ONNX, an open graph format that ONNX Runtime executes on CPU, GPU and mobile, and that TensorRT can compile for NVIDIA GPUs. TensorFlow and Keras models export to SavedModel for TF Serving or convert to LiteRT (formerly TensorFlow Lite) for phones and microcontrollers. scikit-learn pipelines are usually serialized with joblib or skops and served in a Python process, or converted with skl2onnx when a non-Python runtime is required. Large language models typically use specialized inference servers such as vLLM or TGI that implement batching and paged KV caches.
Two rules avoid most export pain: keep preprocessing inside the exported graph (or in one shared library), and test the exported model against the original on a fixed batch, comparing outputs within a small numeric tolerance.
Every framework offers the same ladder. Data parallelism (PyTorch DDP, tf.distribute.MirroredStrategy) replicates the model on each GPU and averages gradients; it is the first and simplest step. When the model plus optimizer state no longer fits on one device, sharded data parallelism (PyTorch FSDP, DeepSpeed ZeRO) splits parameters, gradients and optimizer states across devices. The largest models add tensor and pipeline parallelism (Megatron-LM style) to split individual layers or groups of layers. Hugging Face Accelerate, PyTorch Lightning and Keras 3's distribution API wrap these behind configuration so the training script changes little. JAX expresses the same ideas through sharding annotations compiled by XLA.
Before scaling out, use mixed precision (bf16 where supported), gradient accumulation for larger effective batches, and activation checkpointing; together they often make a single GPU sufficient for fine-tuning medium-size models.
| Need | Common tools |
|---|---|
| Gradient-boosted trees | XGBoost, LightGBM, CatBoost (all with scikit-learn-compatible APIs) |
| Experiment tracking | MLflow, Weights & Biases, TensorBoard |
| Hyperparameter search | Optuna, Ray Tune, KerasTuner |
| Data loading at scale | PyTorch DataLoader, tf.data, Hugging Face Datasets (Arrow-backed) |
| Parameter-efficient fine-tuning | Hugging Face PEFT (LoRA, QLoRA), bitsandbytes |
| Research-style functional ML | JAX with Flax or Equinox, Optax |
Version pinning matters across this stack: CUDA, cuDNN, the framework build and driver versions must agree. Use the framework's official install selector or containers (NVIDIA NGC, official framework images) rather than mixing pip wheels by hand. See the NVIDIA GPU installation notes for a worked local setup.
Further reading: Paszke, A. et al. (2019), "PyTorch: An imperative style, high-performance deep learning library"; Abadi, M. et al. (2016), "TensorFlow: A system for large-scale machine learning"; Pedregosa, F. et al. (2011), "Scikit-learn: Machine learning in Python" (JMLR); Wolf, T. et al. (2020), "Transformers: State-of-the-art natural language processing"; Rajbhandari, S. et al. (2020), "ZeRO: Memory optimizations toward training trillion parameter models".