Edit: from the Tabular CEO announcement
Databricks reached out to me and proposed a collaboration that could bring Iceberg and Delta closer together [...] I’m excited to have the opportunity to work with Databricks and the broader Delta community to build better table formats together.
So it seems they are going for the latter.Either way, I just want to know which format to pick. I've been chief data engineer at my current company for about a year and would like to be able to move off of plain parquet files in my lake but I'm not sure what table format to choose.
I have similar questions about the future of Delta Lake, but not really about the future of Iceberg, that's what the Apache Foundation is for after all. There are enough large enterprise players relying on this (Apple, Netflix, ...) to keep the project going for a while.
Yesterday we announced Polaris specifically so (1) customers don't get locked into a catalog; (2) people know Snowflake works with AWS, Azure, Confluent, etc.
1: https://www.snowflake.com/blog/introducing-polaris-catalog/
[1] https://www.cnbc.com/2024/06/04/databricks-is-buying-data-op...
But yes, this is definitely bad for Snowflake, Databricks can position itself as a very strong competitor with this move and moving more towards Iceberg.
The big difference - innodb got like 3m, Tabular 1bn!