Project Nessie is an open-source “Git-for-data” catalog for Apache Iceberg, started by Dremio. It treats data tables the way Git treats source code — commits, branches, tags, and merges — layered on top of an Iceberg metastore. Nessie makes experimentation, isolation, and rollback first-class operations instead of one-off engineering exercises.
Below: how Nessie stores versions, the branch-validate-merge workflow, a Spark SQL example, and how it compares with other lakehouse catalogs.
etl-2026-04-25) that points at the current state of every table; load data, validate it, then merge into main or discard.SELECT * FROM table @ branch:dev.Traditional lakehouse releases mix data ingestion with data publication — a partial load is visible to consumers immediately. Nessie inverts this: ingestion happens on a feature branch, validation and QA run against that branch, and only a successful merge exposes the change to the production view. The data team gets the same review-and-promote workflow software engineers have had since the late 2000s.
main for ad-hoc exploration.For an Iceberg table, the only mutable piece of state is the pointer to the table's current metadata file. An ordinary catalog stores one such pointer per table. Nessie instead stores a commit log: each commit records the set of table pointers that changed, and a branch or tag is a named reference to a commit. Because a commit can move pointers for several tables at once, a multi-table change becomes atomic. Engines talk to Nessie through its own API or the Iceberg REST catalog protocol, specifying the reference to read or write. The version store behind the server is pluggable, with backends such as JDBC databases, RocksDB for single-node use, and cloud key-value stores.
The core workflow mirrors feature branches in Git. A job creates a branch from main, writes new snapshots of one or more tables on that branch, runs data-quality checks against the branch, and merges only if the checks pass. If they fail, the branch is dropped and consumers never saw the bad data. Merges are applied per table: if main changed a different table meanwhile, the merge succeeds; if the same table changed on both sides, Nessie reports a conflict rather than silently combining Iceberg snapshots.
-- Spark session configured with the Nessie catalog named "nessie"
CREATE BRANCH IF NOT EXISTS etl_2026_10_01 IN nessie FROM main;
USE REFERENCE etl_2026_10_01 IN nessie;
INSERT INTO nessie.sales.orders SELECT * FROM staging_orders;
MERGE INTO nessie.sales.customers t USING staging_customers s
ON t.id = s.id
WHEN MATCHED THEN UPDATE SET *
WHEN NOT MATCHED THEN INSERT *;
-- validation query on the branch
SELECT count(*) FROM nessie.sales.orders WHERE amount < 0;
-- publish both tables atomically
MERGE BRANCH etl_2026_10_01 INTO main IN nessie;
CREATE TAG month_end_2026_09 IN nessie FROM main;These statements come from the Nessie Spark SQL extensions, which must be added to the Spark session alongside the Iceberg extensions. The tag gives auditors and ML pipelines a stable name for the state of every table at month end.
| Catalog | Distinctive feature | Fits when |
|---|---|---|
| Nessie | Git-style branches, tags, multi-table commits | Write-audit-publish pipelines, reproducible snapshots |
| Apache Polaris | Iceberg REST catalog with RBAC | Vendor-neutral Iceberg governance |
| Unity Catalog | Governance across tables, files, models | Databricks-centred platforms |
| Hive Metastore / AWS Glue | Widest legacy engine support | Existing Hive-era estates |
main and produce merge conflicts; keep ETL branches short.See Apache Iceberg for the table format Nessie versions, the Catalogs overview, and Trino for querying Nessie-managed tables.