iceberg-go

er efficient Parquet flush/roll as the target file size is reached, all the usual Iceberg transforms supported: identity, bucket, truncate, year/month/day/hour Arrow-based write for stable memory & fast columnar handling and why we’re bullish on Go for building our platform - OLake? the runtime’s concurrency model makes it straightforward to coordinate partition writers, batching, and backpressure. small static binaries → easy to ship edge and sidecar ingestors. great ops story (observability, profiling, and sane resource usage) which is a big deal when you’re replicating at high rates. where this helps right now: building micro-ingestors that stream changes from DBs to Iceberg in Go. edge or on-prem capture where you don’t want a big JVM stack. teams that want cleaner tables (fewer tiny files) without a separate compaction job for every write path. For data teams still worried about Go, we have our case study helps you : check the benchmarks we’re hitting thanks to the language’s lightweight model See numbers here: https://olake.io/docs/benchmarks If you’re experimenting with Go + Iceberg, we’d love to collaborate as we believe in open source :) repo: https://github.com/datazip-inc/olake/

ataframes to it! https://github.com/turbolytics/sql-flow Arrow is quite spectacular and it's cool to see the industry moving to standardize on it as a dataframe. For example, Clickhouse python also support arrow-based insertion: https://sql-flow.com/docs/tutorials/clickhouse-sink This makes the glue code trivial to sink into these different systems as long as arrow is used.