Show HN: Open-Source C++ Apache Iceberg Client with Write Support (opens in new tab)

(github.com)

8 pointsjovezhong1y ago3 comments

Existing OSS C++ projects like ClickHouse and DuckDB support reading from Iceberg tables. Writing requires Spark, PyIceberg, or managed services.

In this PR https://github.com/timeplus-io/proton/pull/928, we are open-sourcing a C++ implementation of Iceberg integration. It's an MVP, focusing on REST catalog and S3 read/write(S3 table support coming soon). You can use Timeplus to continuously read data from MSK and stream writes to S3 in the Iceberg format. No JVM. No Python. Just a low-overhead, high-throughput C++ engine. Docker/K8s are optional. Demo video: https://www.youtube.com/watch?v=2m6ehwmzOnc

Help us improve the code to add more integrations and features. Happy to contribute this to the Iceberg community. Or just roast the code. We’ll buy the virtual coffee.

3 comments

tingfirst1y ago

Pretty cool to see a C++ R/W Iceberg client without dependency, and even better open-sourced. The pipeline is all about processing and routing, ideally, to open and flexible destination with no lock-in and long-term retention. Writing into Apache Iceberg is becoming critical to give users real control, rather than into specific data warehouses or lakehouses that are hard to move out.

ZiliangXK1y ago

Look like other folks are doing similar things, wondering any benchmark results so far compared with Java implementation

gangtao1y ago

can you provide more information about the other similar project, would like to take a look

j / k navigate · click thread line to collapse