undefined | Better HN

0 pointsOutOfHere1y ago0 comments

For the longest time, searching for Ballista linked to its old archived repo that didn't even have a link to the new repo. There was no search result for the new repo. This misled people into thinking that Ballista is a dead project but it wasn't. It wasted so much opportunity.

I don't think it's a fair criticism of Ballista to say that it failed in any way. It just looks to need substantial effort to bring it on par with Spark. The performance benefits are meaningful. Ballista can then not only take the crown from Spark, but also revalidate Rust as a language.

0 comments

andygrove1y ago

I wish I'd known about the search issue.

I do see a new opportunity for Ballista. By leveraging all of the Spark-compatible operators and expressions being built in Comet, it would be able to support a wider range of queries much more quickly.

Ballista already uses protobuf for sending plans to executors and Comet accepts protobuf plans (in a similar, but different format).

OutOfHereOP1y ago

Did Databricks sponsor Comet?

andygrove1y ago

No.

j / k navigate · click thread line to collapse

0 comments

andygrove1y ago

I wish I'd known about the search issue.

Ballista already uses protobuf for sending plans to executors and Comet accepts protobuf plans (in a similar, but different format).

OutOfHereOP1y ago

Did Databricks sponsor Comet?

andygrove1y ago

No.

j / k navigate · click thread line to collapse