Views, committed to Iceberg.
Write any view to an Apache Iceberg table in your own bucket. Each commit is an atomic snapshot that Spark, Trino and Snowflake can read.
Overview
How it works.
Basalt writes Parquet data files to your bucket and commits them through a REST, Glue or Nessie catalog. Each commit records the source offsets in snapshot properties, so a retried commit is detected and skipped.
Setup
Set it up in three steps.
01
Add the destination
Point Basalt at your catalog and give it write access to the warehouse bucket. Basalt creates the table with the view's schema and partition spec.
# commit the orders view to a Glue-catalog table basalt sink add orders --iceberg glue://lake.orders --view orders_enriched
✓ table lake.orders created · format v2
✓ snapshot 1 committed · 128 files · 1.9 GB
02
Shape it with a view
Write plain SQL over the source. Basalt keeps the view current with every record and tracks lineage back to the topic, partition and offset.
CREATE MATERIALIZED VIEW revenue_by_min AS SELECT date_trunc('minute', ts) AS minute, currency, sum(amount) AS revenue, count(*) AS payments FROM payments WHERE status = 'captured' GROUP BY 1, 2;
03
Read it from anywhere
Every view is an endpoint the moment it exists. Call it over REST, connect with any Postgres client, or use the SDK from your app.
curl https://api.basalt.dev/v1/views/revenue_by_min?limit=5 \ -H "Authorization: Bearer $BASALT_TOKEN"
✓ 200 OK · 5 rows · served from eu-north-1 in 3.1 ms
Included
What you get.
01
Atomic commits
Every batch is one snapshot. Readers never see a partial write.
02
Any catalog
Works with the REST catalog, AWS Glue, Nessie and Polaris.
03
File compaction
Small files are rewritten on a schedule so query engines scan fewer objects.
04
Time travel
Query any past snapshot by ID or timestamp from your engine of choice.
Related
