Data architecture decides where data lives, how it flows and who owns it. This section covers the core patterns and how to choose between them.
Data architecture is the set of decisions about where data lives, how it moves, who owns it and how it is served. Individual tools change every few years, but the underlying questions are stable: should this workload run on an operational database or an analytical one? Is the system of record a warehouse, a lake or a lakehouse? Do we process in batches, in streams, or both? Is the platform run by one central team or owned by business domains?
The pages in this section each address one of those questions. Together they describe the reference platform above: operational sources, ingestion, storage and compute, serving, and a governance layer across everything.
Start with OLTP vs OLAP, the most fundamental split: transactional systems optimise for many small reads and writes with strong consistency, analytical systems for large scans and aggregations. Next, Data Warehouse vs Data Lake vs Lakehouse covers the main options for the analytical side and how open table formats merged the first two. Lambda and Kappa Architectures addresses time: how to serve both complete historical answers and fresh results. Data Mesh addresses organisation: who owns datasets and how decentralised teams publish them as products. These are not mutually exclusive; a company can run a lakehouse, a Kappa-style streaming layer and a mesh operating model at the same time.
Beyond these core pages, the neighbouring sections go deeper into the building blocks: storage formats explain how bytes are laid out in files, the lakehouse section covers table formats, catalogs and query engines, and the pipelines section covers ingestion and orchestration. Read the architecture pages for the shape of the system, then those sections for how each layer is built.
| Question | Options | Leaning one way | Page |
|---|---|---|---|
| Transactional or analytical? | OLTP, OLAP, HTAP | Row-level updates with constraints point to OLTP; scans and aggregates to OLAP | OLTP vs OLAP |
| Where does analytical data live? | Warehouse, lake, lakehouse | Mostly SQL BI on structured data: warehouse. Mixed ML and BI on open files: lakehouse | Warehouse vs Lake vs Lakehouse |
| How fresh must results be? | Batch, Lambda, Kappa | Hours are fine: batch. Seconds with replayable history: Kappa | Lambda and Kappa |
| Who owns the data? | Central team, data mesh | Many domains, central team a bottleneck: mesh | Data Mesh |
Architecture choices should follow from requirements you can state: latency targets, data volumes, the number of teams producing data, regulatory constraints and the skills on hand. A small team with one warehouse and nightly batch jobs is often better served than one running five fashionable systems it cannot operate.