undefined | Better HN

0 pointsgeoduck144y ago0 comments

I'd like to add some points: Ive used Snowflake for several years. Snowflake works with structured and semi-structured data (think spreadsheets and JSON). I've never tried working with pics or videos - and I'm not sure it would make sense to do that.

I've evaluated Databricks. It works with the above mentioned structured and semi-structured data. I also suspect it could process unstructured data. My understanding is that it runs Python (and some others), so you can do any "Python stuff, but in the cloud, and on 1000s of computers"

0 comments

629514134y ago

Databricks used to be an Apache Spark as a service company. And Spark is a predominantly Scala code base. PySpark is just a Python binding for the real engine popular in ML circles. In the last couple of years the Databricks platform migrated from open-source Spark to a new proprietary engine written in C++.

tomnipotent4y ago

You're referring to PySpark, which still does all the heavy lifting in the JVM.

j / k navigate · click thread line to collapse

0 pointsgeoduck144y ago0 comments

0 comments

629514134y ago

tomnipotent4y ago

You're referring to PySpark, which still does all the heavy lifting in the JVM.

j / k navigate · click thread line to collapse