Kinesis shards, read in order.
Connect a stream with an IAM role. Basalt reads each shard with enhanced fan-out and checkpoints by sequence number.
Overview
How it works.
Basalt registers a stream consumer and subscribes to each shard with SubscribeToShard. Records are written in sequence order, and the checkpoint advances only after the write is durable. Shard splits and merges are followed automatically.
Setup
Set it up in three steps.
01
Add the source
Create an IAM role Basalt can assume with an external ID. Pass the stream ARN and Basalt registers its own fan-out consumer.
# read the events stream with enhanced fan-out basalt source add events --kinesis arn:aws:kinesis:us-east-1:4021:stream/events --role basalt-reader
✓ consumer basalt-events registered · EFO
✓ 16 shards streaming · iterator age 120 ms
02
Shape it with a view
Write plain SQL over the source. Basalt keeps the view current with every record and tracks lineage back to the topic, partition and offset.
CREATE MATERIALIZED VIEW revenue_by_min AS SELECT date_trunc('minute', ts) AS minute, currency, sum(amount) AS revenue, count(*) AS payments FROM payments WHERE status = 'captured' GROUP BY 1, 2;
03
Read it from anywhere
Every view is an endpoint the moment it exists. Call it over REST, connect with any Postgres client, or use the SDK from your app.
curl https://api.basalt.dev/v1/views/revenue_by_min?limit=5 \ -H "Authorization: Bearer $BASALT_TOKEN"
✓ 200 OK · 5 rows · served from eu-north-1 in 3.1 ms
Included
What you get.
01
Enhanced fan-out
A dedicated 2 MB/s per shard pipe, so Basalt never competes with your other readers.
02
Sequence checkpoints
Sequence numbers are stored with the write. A restart resumes at the exact record.
03
Reshard handling
Splits and merges are followed in parent-child order without losing ordering.
04
KPL aggregation
Aggregated records from the Kinesis Producer Library are unpacked automatically.
Related
