Views, written to BigQuery.
Sync any view to a BigQuery table through the Storage Write API. Tables are partitioned by time and queryable within seconds.
Overview
How it works.
Basalt writes to committed-type streams with explicit row offsets, so BigQuery rejects any duplicate append. Tables are partitioned by the view's time column and clustered by its key. Schema changes are applied before the next batch.
Setup
Set it up in three steps.
01
Add the destination
Grant a service account BigQuery Data Editor on the dataset. Basalt creates the table, its partitioning and its clustering.
# write the sessions view to BigQuery basalt sink add sessions --bigquery acme-prod.analytics --view sessions_live
✓ table sessions_live created · partitioned by DAY(started_at)
✓ committed stream open · 41k rows/s
02
Shape it with a view
Write plain SQL over the source. Basalt keeps the view current with every record and tracks lineage back to the topic, partition and offset.
CREATE MATERIALIZED VIEW revenue_by_min AS SELECT date_trunc('minute', ts) AS minute, currency, sum(amount) AS revenue, count(*) AS payments FROM payments WHERE status = 'captured' GROUP BY 1, 2;
03
Read it from anywhere
Every view is an endpoint the moment it exists. Call it over REST, connect with any Postgres client, or use the SDK from your app.
curl https://api.basalt.dev/v1/views/revenue_by_min?limit=5 \ -H "Authorization: Bearer $BASALT_TOKEN"
✓ 200 OK · 5 rows · served from eu-north-1 in 3.1 ms
Included
What you get.
01
Stream offsets
Appends carry explicit offsets, so retries are rejected instead of duplicated.
02
Time partitioning
Tables partition by hour or day and cluster on the view key for cheaper scans.
03
Schema updates
New view columns are added to the table before rows that use them arrive.
04
Cost control
Writes use the Storage Write API, with 2 TB a month free before billing starts.
Related
