Apache XTable

Apache XTable (formerly OneTable, originated at Onehouse) is an open-source interoperability layer that translates table metadata between Apache Hudi, Apache Iceberg, and Delta Lake. The data files (Parquet) stay in place; XTable rewrites only the metadata so the same physical dataset can be read as any of the three formats by any compatible engine. It sidesteps the “format wars” by letting teams pick formats per consumer rather than per producer.

1. How It Works

One Copy of Data, Three Sets of Table Metadata Parquet data fileswritten once, sharedHudi timeline.hoodie/Delta log_delta_log/Iceberg metadatametadata/*.jsonXTable reads the source format metadata (yellow) and writes equivalent metadata for the targets.Data files are not copied or rewritten; all three formats point at the same Parquet files.

Delta Lake, Apache Iceberg and Apache Hudi all store table data as Parquet files on object storage. What makes each of them a "format" is the metadata layer next to those files: Delta's JSON and checkpoint log, Iceberg's metadata JSON, manifest lists and manifests, and Hudi's timeline. That metadata records which files belong to each snapshot, the schema, partitioning and statistics. XTable exploits this: since the data files are already compatible, it only needs to translate the metadata. A sync reads the commits of the source format and produces the equivalent commits for each target format in the same table directory.

2. Key Features:

3. Why It Matters:

Many organizations end up multi-format by accident: data engineers write Hudi from Spark Structured Streaming, while analysts query through Snowflake or BigQuery (which have stronger Iceberg support), and data scientists train on Databricks (Delta-native). XTable avoids forcing a re-platform — the producer keeps its preferred format and downstream consumers see the format they prefer.

4. Use Cases:

5. Sync Process

Incremental Sync Run Read sourcelatest commitsDiffsince last syncMap schematypes, partitionsWrite targetnew snapshot/commitRecord statesync checkpointRun after each source commit or on a schedule; readers of the target formats see the new snapshot once the target commit lands.

XTable supports full and incremental sync. On the first run it translates the full current snapshot. Later runs read only the source commits made since the last sync and translate them into new target snapshots, which keeps sync cost proportional to change rather than table size. Schema changes and partition specifications are mapped between formats where an equivalent exists. Because the target metadata is produced after the fact, readers of the target formats lag the source by one sync interval; the source format remains the single writer.

6. Configuration Example

XTable is distributed as a Java utilities bundle driven by a YAML file that names the source format, target formats and the tables to sync:

sourceFormat: DELTA
targetFormats:
  - ICEBERG
  - HUDI
datasets:
  - tableBasePath: s3://lake/sales/orders
    tableName: orders
  - tableBasePath: s3://lake/sales/customers
    tableName: customers
    partitionSpec: region:VALUE
# Build or download the bundled utilities jar from the Apache XTable project, then:
java -jar xtable-utilities-bundled.jar --datasetConfig xtable_config.yaml

After the run, the orders directory contains a metadata/ folder, so an Iceberg reader such as Snowflake, Trino or Athena can register it (for example through an Iceberg catalog or by pointing at the metadata file) while Spark jobs continue to write Delta. For Iceberg targets, the new metadata must also be registered with whatever catalog your engines use; XTable can be configured with catalog settings for this.

7. When to Use It

SituationRecommendation
Writers use one format, some engines can only read anotherXTable sync on a schedule or after commits
Migrating from Hudi or Delta to IcebergXTable to run both side by side, then cut over writers
Need multiple concurrent writers in different formatsNot supported: pick one writer format
Delta-only shop wanting Iceberg readersAlso consider Delta UniForm, which generates Iceberg metadata natively on write
Single engine, single formatNot needed

8. Pitfalls