Summarization produces a shorter text that preserves the key information of a source document. There are two families: extractive methods select sentences from the source, and abstractive methods generate new text, typically with an encoder-decoder such as BART or T5, or with a decoder-only LLM.
The hard parts are not fluency but coverage (did it keep what matters?), faithfulness (is every claim supported by the source?) and length (documents that exceed the context window). Part of Natural Language Processing.
Extractive summarization scores each sentence and returns the top-k, often re-ranked with Maximal Marginal Relevance (MMR) to avoid redundancy: MMR = argmax [λ · sim(s, doc) − (1 − λ) · max sim(s, selected)]. Classic scorers are TextRank (PageRank over a sentence-similarity graph) and supervised sentence classifiers on top of a BERT-style encoder.
Abstractive summarization treats the task as conditional generation P(summary | document). Models are pretrained with denoising objectives (BART reconstructs corrupted text; PEGASUS masks whole "gap sentences") and then fine-tuned on datasets such as CNN/DailyMail or XSum. Instruction-tuned LLMs do it zero-shot, with control over length, audience and format through the prompt.
# Minimal extractive baseline: TF-IDF sentence centrality (TextRank-style)
import numpy as np, re
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
def extractive(text, k=2):
sents = [s.strip() for s in re.split(r"(?<=[.!?])\s+", text) if s.strip()]
X = TfidfVectorizer(stop_words="english").fit_transform(sents)
sim = cosine_similarity(X)
np.fill_diagonal(sim, 0)
# power iteration for PageRank on the similarity graph
P = sim / np.maximum(sim.sum(1, keepdims=True), 1e-9)
r = np.ones(len(sents)) / len(sents)
for _ in range(50):
r = 0.15 / len(sents) + 0.85 * P.T @ r
top = sorted(np.argsort(-r)[:k]) # keep original order
return " ".join(sents[i] for i in top)This is worth keeping as a baseline: it is deterministic, fast, needs no GPU, and cannot hallucinate. The lead-3 baseline (first three sentences) is also surprisingly strong on news, because journalists front-load the key facts; any abstractive model should beat both before you ship it.
# pip install transformers torch rouge-score
from transformers import pipeline
from rouge_score import rouge_scorer
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
article = """The city council voted on Tuesday to expand the bike-lane network by
40 kilometres over three years. Supporters said the plan would cut traffic and
emissions; opponents raised concerns about parking and cost. Construction of the
first segment is scheduled to begin next spring."""
out = summarizer(article, max_length=60, min_length=15,
num_beams=4, no_repeat_ngram_size=3, do_sample=False)
summary = out[0]["summary_text"]
print(summary)
ref = "The council approved a 40 km bike-lane expansion over three years."
scorer = rouge_scorer.RougeScorer(["rouge1", "rouge2", "rougeL"], use_stemmer=True)
print(scorer.score(ref, summary))no_repeat_ngram_size suppresses the repetition loops seq2seq models fall into; num_beams with a length penalty trades brevity against coverage. BART-large-CNN has a 1,024-token input limit, so anything longer must be truncated or chunked.
When the input exceeds the context window, three strategies dominate:
Chunk on semantic boundaries (sections, paragraphs) with a small overlap rather than fixed token counts.
ROUGE measures n-gram overlap with reference summaries and is the standard benchmark metric, but it rewards lexical similarity, not correctness. BERTScore compares contextual embeddings and tolerates paraphrase. Neither detects a fluent sentence that contradicts the source.
For faithfulness, use NLI-based checks (does the source entail each summary sentence?), QA-based checks (generate questions from the summary, answer them from the source, compare), or an LLM judge with a rubric. In production, sample outputs for human review and track the rate of unsupported claims explicitly.
Suppose a document has three candidate sentences with similarity to the whole document of s1 = 0.80, s2 = 0.75, s3 = 0.60, and s1 and s2 say nearly the same thing (similarity 0.90 to each other), while s3 has similarity 0.20 to s1. With λ = 0.7, the first pick is s1 because nothing is selected yet. For the second pick, s2 scores 0.7·0.75 − 0.3·0.90 = 0.255, while s3 scores 0.7·0.60 − 0.3·0.20 = 0.36. MMR therefore picks s3: slightly less central, but it adds new information. Higher λ favors relevance, lower λ favors diversity; 0.5–0.8 is a typical range.
Map-reduce summarizes each chunk independently (in parallel) and then summarizes the summaries, recursively if needed. It scales to any length and is fast, but cross-chunk references ("as described above") and the overall argument can get lost. Refine walks the chunks in order and updates a running summary, which preserves narrative flow but is sequential, slower, and biased toward whatever the draft already says. A third option with long-context LLMs is to fit the whole document and ask for the summary directly; studies of long-context models report that information in the middle of long inputs is used less reliably than information at the start or end (Liu et al., 2023, "Lost in the Middle"), so long single-pass summaries still benefit from structure such as section headings in the prompt or an outline-first approach.
Chunk on semantic boundaries (sections, paragraphs), not fixed character counts, and carry a small amount of context (document title, section heading) into each chunk prompt.
ROUGE measures word overlap with a reference and says little about whether claims are true. Abstractive models, especially on XSum-style one-sentence summaries, frequently add facts not in the source (Maynez et al., 2020). Practical checks decompose a summary into atomic claims and verify each against the source, using a natural language inference model (as in SummaC) or a question-answering approach (QAGS, QuestEval), or an LLM judge with the source in context. Report the fraction of supported claims alongside coverage.
To reduce hallucination: instruct the model to use only the provided text, ask for quotes or source spans per bullet, keep temperature low, and prefer extractive or "extract-then-rewrite" pipelines for high-stakes content such as medical or legal documents. Numbers, names and dates deserve exact-match verification against the source.
Related: Encoder-Decoder Transformers, Prompt Engineering.