undefined | Better HN

0 pointssmabie5y ago0 comments

Pandas/python is amazingly prevalent at trading firms. And everyday, we bitch about the performance, we bitch about the stupid API, we bitch about the GIL, the lack of expressiveness. The list goes on and on. But for some braindead reason, we never switch to Julia. It's masochistic.

0 comments

ivirshup5y ago

I do think Julia is a far better language for numerics than python, but compared to DataFrames.jl, pandas can be quite fast. I know, "but it's easier to make it faster in Julia". Last I checked `sort(df, :col)` was significantly slower than `df[sortperm(df[:col])]`. Someone actually has to go through and make these libraries fast.

Second issue, in my field (bioinformatics) the script is still a pretty common unit of code. Without cached compilation being a simple flag, Julia often is slower.

smabieOP5y ago

Yeah, that's a good point. DataFrames.jl starts to really shine what the cookie cutter pandas functions arent adequate for what you need to do. DataFrames.jl can certainly be slower in some cases, but you should expect a consistent level of performance no matter what you do. This is a farcry from Pandas, which tanks by large factors when you start calling Python code vs C code.

In regards to Julia's compilation problem, you can use https://github.com/JuliaLang/PackageCompiler.jl to precompile an image, allowing you to avoid paying the JIT performance penalty over and over again.

j / k navigate · click thread line to collapse

0 comments

ivirshup5y ago

Second issue, in my field (bioinformatics) the script is still a pretty common unit of code. Without cached compilation being a simple flag, Julia often is slower.

smabieOP5y ago

j / k navigate · click thread line to collapse