by pkhodiyar on 3/13/25, 12:45 PM with 1 comments
by pkhodiyar on 3/13/25, 12:45 PM
1. Sync Postgres -> AWS S3 in parquet format [JSON dump and Level 1 flattened data support] 2. Sync Postgres -> Iceberg [in a few days, schema evolution support of Iceberg] 3. Sync Postgres -> Local Filestorage with everything we support for S3.
Want to test it out locally? Sync Postgres via OLake -> MinIO (using JDBC catalog) -> Query using Engines that support Iceberg V2 tables [doc launching pretty soon]
We talked with 100s of DE’s to understand their pain point and the two major issues for them were replicating Postgres and MySQL to a lakehouse format. They said, we delivered.
Our source connectors and writers are independent, meaning that after Apache Iceberg writers are being supported (PR - https://github.com/datazip-inc/olake/pull/113), any new connector will be able to dump to Iceberg with minimal changes!