Warehouses, lakes and lakehouses are three answers to the same question: where should analytical data live and how should it be queried? This page traces how each arose, what actually differs underneath, and how to choose.
A data warehouse is a database designed for analytics: data is cleaned and modelled before loading (schema-on-write), stored in a format the engine controls, and queried with SQL. It is fast, consistent and well governed, but storage is tied to one vendor and unstructured data such as images or logs fits poorly.
A data lake is cheap storage — originally HDFS, now object stores such as Amazon S3, Azure Data Lake Storage or Google Cloud Storage — holding files in open formats (Parquet, ORC, JSON, CSV, images). Structure is applied when data is read (schema-on-read). It is flexible and inexpensive but lacks transactions, so concurrent writers corrupt tables, and without discipline it decays into a "data swamp".
A lakehouse keeps the lake's open files on object storage and adds a table format — Delta Lake, Apache Iceberg or Apache Hudi — that records which files make up each table version in a transaction log. That log brings ACID transactions, schema enforcement, time travel and efficient metadata to plain files, so warehouse-style SQL and machine learning can share one copy of the data.
The choice decides where your data physically lives, which engines can read it, how much you pay, and how governance is enforced. Teams that ran a lake for data science and a warehouse for BI often maintained two copies of the same data with separate pipelines that drifted apart. The lakehouse pitch is to remove that duplication. At the same time, cloud warehouses such as Snowflake and BigQuery now read and write Iceberg tables, so the boundary is blurring: the more useful question today is "which open format and catalog, and which engines on top?"
For learners, the three terms are also a good map of the history of data engineering. Most job descriptions, tools and interview questions assume you can explain why each generation appeared, what problem it solved, and what new problem it introduced. Understanding that chain makes vendor marketing much easier to read critically.
Warehouses (Inmon and Kimball designs from the 1990s) ran on expensive appliances where storage and compute scaled together. Hadoop made storage cheap and moved computation to the data, but early Hive tables were just directories: no atomic updates, no deletes. Cloud object storage then separated storage from compute entirely, and table formats added the missing database guarantees on top.
The diagram shows the key structural difference. In a warehouse one engine owns the stack. In a lake any engine can read the files, but nothing coordinates writers. In a lakehouse the table format is the coordination layer: a commit atomically swaps in a new metadata snapshot listing data files, so readers always see a consistent version. Statistics in that metadata (min/max per column per file) let engines skip files, recovering much of a warehouse's query performance.
Separation of storage and compute is now common to all three in the cloud: Snowflake, BigQuery and Redshift RA3 keep data in object storage internally and scale virtual compute clusters independently. The real difference is whether the storage format is open (readable by other engines without export) or proprietary.
Governance differs too. Warehouses provide role-based access, row and column security and auditing inside the engine. On a lake, permissions are on files and buckets, which is too coarse for "analysts may not see the SSN column". Lakehouses push governance into a catalog (Unity Catalog, Apache Polaris, AWS Glue with Lake Formation) that every engine consults, so one policy applies to Spark, Trino and SQL warehouses alike.
Lake files alone cannot do an atomic upsert; with a table format they can. This PySpark snippet (Delta Lake 3.x, pip install delta-spark) creates a table, merges changes and reads an older version.
from pyspark.sql import SparkSession
from delta import configure_spark_with_delta_pip
builder = (SparkSession.builder.appName("lakehouse-demo")
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog",
"org.apache.spark.sql.delta.catalog.DeltaCatalog"))
spark = configure_spark_with_delta_pip(builder).getOrCreate()
spark.createDataFrame([(1, "alice", 10.0), (2, "bob", 20.0)],
"id INT, name STRING, balance DOUBLE") \
.write.format("delta").mode("overwrite").save("/tmp/lh/accounts")
spark.createDataFrame([(2, "bob", 25.0), (3, "carol", 5.0)],
"id INT, name STRING, balance DOUBLE") \
.createOrReplaceTempView("updates")
spark.sql("""
MERGE INTO delta.`/tmp/lh/accounts` t
USING updates u ON t.id = u.id
WHEN MATCHED THEN UPDATE SET *
WHEN NOT MATCHED THEN INSERT *
""")
spark.read.format("delta").load("/tmp/lh/accounts").show() # 3 rows, bob = 25
spark.read.format("delta").option("versionAsOf", 0) \
.load("/tmp/lh/accounts").show() # original 2 rowsLook inside /tmp/lh/accounts: Parquet data files plus a _delta_log/ folder of JSON commits. That folder is the whole difference between a lake directory and a lakehouse table.
Most lakehouses organise tables in a medallion layout. Bronze keeps raw ingested data for replay, silver holds cleaned and de-duplicated entities, and gold contains business-level aggregates and feature tables. Quality gates between layers are where tools such as Great Expectations or dbt tests run.
| Aspect | Warehouse | Lake | Lakehouse |
|---|---|---|---|
| Schema | On write, enforced | On read, optional | On write, enforced, evolvable |
| Storage format | Proprietary (mostly) | Open files | Open files + open table format |
| Transactions | Full ACID | None | ACID per table |
| Data types | Structured, semi-structured | Anything | Anything; tables for structured |
| Main users | Analysts, BI | Data scientists, engineers | Both |
| Performance tuning | Mostly automatic | Manual (file layout) | Compaction, clustering, statistics |
| Governance | In-engine, mature | Bucket/file level | Catalog-based, maturing |
| Best when | SQL-centric team, managed simplicity | Cheap archive, raw ML data | Mixed BI + ML, multiple engines, avoid lock-in |
Rules of thumb: a small analytics team with mostly SQL workloads is usually happiest with a managed warehouse. A lake alone remains fine as a landing zone and archive. When you need several engines on the same data, large ML workloads next to BI, or freedom to change vendors, pick a lakehouse table format and a catalog first, then choose engines.
OPTIMIZE, Iceberg rewrite_data_files).VACUUM, expire_snapshots) or storage grows without bound.