I tried Julia for the first time last week and it was great.
I've been playing with the idea of defining a hash function for lists that can be composed with other hashes to find the hash of the concatenation of the lists. I tried to do this with matrix multiplication of the hash of each list item, but with integer mod 256 elements random matrices are very likely to be singular and after enough multiplications degenerates to the zero matrix. However, with finite field (aka Galois fields) elements, such a matrix is much more likely to be invertible and therefore not degenerate. But I don't really know anything about finite fields, so how should I approach it? Here's where Julia comes in: with some help I was able to combine two libraries, LinearAlgebraX.jl which has functions for matrices with exact elements, with GaloisFields.jl which implements many types of GF, and wrote up a working demo implementation of this "list hash" idea in a Pluto.jl [2] notebook and published it [0] (and a question on SO [1]) after a few days without having any Julia experience at all. Julia seems pretty approachable, has great libraries, and is very powerful (I was even able to do a simple multithreaded implementation in 5 lines).
[0]: https://blog.infogulch.com/2021/07/15/Merklist-GF.html
[1]: https://crypto.stackexchange.com/questions/92139/using-rando...
I would never use it for producing software meant for distribution, but people mainly hate on it because it's 'cool', without realising that it excels at what it does. I fucking love matlab.
I am a python developer who has dabbled with Julia but it never stuck for me.
I think Julia was built by academics for other academics running innovative high performance computing tasks. It excels at the intersection of 1) big data, so speed is important, and 2) innovative code, so you can't just use someone else's C package. Indeed, Julia's biggest successful applications outside academica closely resemble an academic HPC project (eg Pumas). I think it will continue to have success in that niche. And that's not a small niche! Maybe it's enough to support a billion dollar company.
But most of us in industry are not in that niche. Most companies are not dealing with truly big data, on our scale, it is cheaper to expand the cluster than it is to rewrite everything in Julia. Most who ARE dealing with truly big data, do not need innovative code; basic summary statistics and logistic regression will be good enough, or maybe some cloud provider's prepackaged turn key neural nets scale out training system if they want to do something fancy.
I think for Julia to have an impact outside of academia (and academia-like things in industry) it will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen? Maybe! I hope so! The world would be better off with more cool data science packages.
But I think the sales pitch of "it's like Pandas and scikit but faster!" is not going to win many converts. So is Jax, Numba, Dask, Ray, Pachyderm, and the many other attempts within the Python community of scaling and speeding Python, that require much less work and expense on my part for the same outcome.
Again, congrats to the team, I will continue to follow Julia closely, and I'm excited to see what innovative capabilities come out of the Julia ecosystem for data scientists like me.
The Two Language Problem makes this more likely than one might think. Those high level python packages that plaster over python's many mediocrities have to be written and maintained by someone, and while extremistan bears the brunt of the pain and has done a remarkable job shielding mediocristan, it's extremistan that gets to decide which language to use for the next killer app.
Of course, python has more inertia than god, so it won't go quietly.
I think this is a good deciding factor. Not just for “big data”. In my experience, with its combination of flexibility and raw speed, Julia makes implementing new algorithms (from scratch) a breezy experience. The strong research/academic presence in the community also helps towards encouraging decent Julia libraries for a lot of cutting edge work.
So if you are working in an area where that could make a significant difference, it’s an excellent reason to use Julia.
> will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen?
If enough cutting-edge work happens in Julia, it’s likely that a few great tools/platforms will emerge from that. We’re already seeing that in the scientific modeling ecosystem (as an example), with Differential Equations infrastructure and PumasAI.
On the other hand it makes function discovery almost impossible [1]. Combined with the way how 'using' keyword exports a predefined subset of functions, this makes the language doomed from larger adoption outside of academia at least as long as there is no superb autocompletion and IDE support.
[1] https://discourse.julialang.org/t/my-mental-load-using-julia...
Why's that? What features or lack thereof of Julia contribute to that experience?
If a major company would pick up Flux.jl and fill out the ecosystem that would be AMAZING.
PyTorch and Tensorflow feel like duct tape and chewing gum all day, every day.
1. https://julialang.org/blog/2019/02/julia-entities/#julia_com...
Happy to be wrong here.
What products/services does Julia Computing sell to justify that Series A? The article doesn't mention anything.
Although the company website lists some "products," there are no price tags or subscription plans attached to anything.
And even if there are revenue-generating products/services on the horizon, how will the company protect itself from smaller, more nimble competitors that don't have a platform obligation to fulfill?
How is this not another Docker? Don't get me wrong, both Julia and Docker are amazing, but have we entered the phase of the VC-funded deliberate non-profit?
That's the entire second paragraph of the article. JuliaHub is a paid cloud computing service for running Julia code and JuliaSim/JuliaSPICE/Pumas are paid domain-specific modeling and simulation products. See also some of the other comments here from Keno[1] and Chris Rackauckas[2]:
What we've noticed is the vast majority of the time it's the data scientist's code that's slow not the actual ML model bit. So allowing them to write very performant code with a dumpy-like syntax and not have to deal with painfully slow pandas, lack of true parallelism, etc. would be a true game changer for ML in industry.
There are a number of components here which enable (what I would call) the expression of more advanced models using Julia's nice compositional properties.
Flux.jl is of course what most people would think of here (one of Julia's deep learning frameworks). But the reality behind Flux.jl is that it is just Julia code -- nothing too fancy.
There's ongoing work for AD in several directions -- including a Julia interface to Enzyme: https://github.com/wsmoses/Enzyme.jl
Also, a new AD system which Keno (who you'll see comment below or above) has been working on -- see Diffractor.jl on the JuliaCon schedule (for example).
Long story short -- there's quite a lot of work going on.
It may not seem like there is a "unified" package -- but that's because packages compose so well together in Julia, there's really no need for that.
The reason for the lag is that Julia has been focusing on general composable compiler, codegen and metaprogramming infrastructure which isn't domain specific, whereas pytorch and friends has been putting lots of dev money into c++ ML focused optimizers.
Once the new compiler stuff is in place, it would be relatively trivial to write such optimizations, in user space, in pure Julia. Then exceeding that would be fairly simple also, plus things like static analysis of array shapes
I'm wondering if most statisticians or researchers deal with data big enough that massively better performance would be enough motivation to switch.
And here Julia is a complete Godsend, since it makes it a joy to implement things from the bottom up.
Sure, you also need a language that already has dataframe libraries, plotting, editor support et cetera, and Julia is lacking behind Python and R in these areas. But Julia's getting there, and at the end of the day, it's a relatively low number of packages that are must-haves.
I remember when they hit $10M and assumed they had continued to grow somewhat. But I didn't realized we'd blown well past $20M — when did that happen?
Most of these arguments are re-hashed on each new Julia post here. A few comments:
For most Julia users, any supposed rift between Python and Julia is not really a big deal -- we can just use PyCall.jl, a package whose interop I've personally used many times to wrap existing Python packages -- and supports native interop with Julia <-> NumPy arrays. Wrapping C is similarly easy -- in fact, easier than "more advanced" languages like Haskell -- whose C FFI only supports passing opaque references over the line.
Ultimately, when arguments between languages in this space arise -- the long term question is the upfront cost, and the maintenance cost for a team. Despite the fact that Julia provides an excellent suite of wrapper functionality, I'm aware that introducing new Julia code into an existing team space suffers from the above issues.
I'm incredibly biased, but I will state: maintaining Julia code is infinitely easier than other uni-typed languages. I've had to learn medium-sized Python codebases, and it is a nightmare compared to a typed language. This total guessing game about how things flow, etc. It really is shocking. Additionally, multiple dispatch is one of those idioms where, one you learn about it, you can't really imagine how you did things before. I'm aware of equivalences between the set of language features (including type classes, multi-method dispatch, protocols or interfaces, etc).
Ultimately, the Julia code I write feels (totally subjectively) the most elegant of the languages I've used (including Rust, Haskell, Python, C, and Zig). Normally I go exploring these other languages for ideas -- and there are absolutely winners in this group -- but I usually come crawling back to Julia for its implementation of multiple dispatch. Some of these languages support abstractions which are strictly equivalent to static multi-method dispatch -- which I enjoy -- but I also really enjoy Julia's dynamism. And the compiler team is working to modularize aspects of the compiler so that deeper characteristics of Julia (even type inference and optimization) can be configured by library developers for advanced applications (like AD, for example). The notion of a modular JIT compiler is quite exciting to me.
Other common comments: time to first plot, no native compilation to binary, etc are being worked on. Especially the latter with new compiler work (which was ongoing before the new compielr work) -- seems feasible within a few quarter's time.
Yes, and even with languages that have these abstractions, they aren't as pervasive as in Julia. Ex. There's a fundamental difference between typclasses and functions in haskell. In julia, there's no such distinction and code is more generic
Stream 1: Build amazing products for particular domains, charge license fees
Stream 2: Build a great SaaS platform for running Julia, charge for compute
Since all of our domain products are built in Julia and often involve significant compute cost for their intended application, hopefully both at the same time :).
It's a bit less of a pure language / infra play and more a product play. Ie, docker / containers was almost a pure infra play in the end. These guys make actual things you can use.
The later sells better into business I think and is less likely to be competed against. Google / AWS et al are generally pretty quick to compete on the infra play level.
hopefully will soon be a dominant force in this direction
Is the Julia business model similar to Redhat or Canonical where they sell consulting services?
Such as:
- the first Julia IDE Juni now depreciated.
- On premise package server
- A wrapper over AWS called JuliaRun that has a nice web interface
- paid "we make a core developer stare at RR traces of your problems"
- FDA approved software for drug development and pharmacokinetics as https://juliacomputing.com/products/pumas/
It is not the programming language that is the product.
1 https://www.hpcwire.com/off-the-wire/julia-joins-petaflop-cl...
2 https://arstechnica.com/science/2020/10/the-unreasonable-eff...
(Not a hater of Julia at all, very much think it's a cool language and an increasingly vibrant ecosystem and have been consistently impressed when Julia devs have spoke at events I've attended)
If Julia can provide both the REPL/debugging experience of a language like Python or MATLAB with a fast enough JIT to use in production it would be an enormous boon to productivity and robustness.
There are a few limiting factors but I don't think they're absolute.
Then there's Matlab, Mathematica, and they are also pretty good but they're closed source/proprietary, so their ecosystem is mostly limited and driven by commercial interests. Nothing wrong with that intrinsically and they're all widely used but it's one way Julia differentiates itself, by making the language open and making money through services.
But indeed, Julia Computing differentiates itself from something like MATLAB or Mathematica by leveraging a strong open source community on which these products are developed. These products add a lot of the details that are generally lacking in the open source space, such as strong adherents to file formats, regulatory compliance and validation, GUIs, etc. which are required to take such a product from "this guy can use it" to a fully marketable product usable by non-computational scientists. I will elaborate a bit more on this at JuliaCon next week in my talk on the release of JuliaSim.
It also pretty much solved my version of the two-language problem, but that means different things to different people so ymmv.
Such a question seems sort of in bad faith (or loaded), since the selling points of Julia have been hammered time and again on HN and elsewhere, and are prominent on its website. It's a 1 minute search to find them, and if someone is already aware that there's this thing called Julia to the point that they think it's made to be "a big deal", they surely have seen them.
So, what could the answer to the question above be? Some objective number that shows Julia is 25.6% better than Java or Rust or R or whatever?
But first, who said it's a "big deal"? It's just a language that has some development action, seems some adoption, and secured a modest fundng for its company. That's not some earth shattering hype (if you want to see that, try to read about when Java was introduced. Or, to a much lesser degree, Ada, for that matter).
You use a language because you've evaluated it for your needs and agree with the benefits and tradeoffs.
Julia is high level and at the same time very fast for numerical computing allowing you to keep a clean codebase that's not a mix of C, C++, Fortran and your "real" language, while still getting most of the speed and easy parallelization. It also has special focus on support for that, for data science, statistics, and science in general. It's also well designed.
On the other hand, it has slow startup/load times, incomplete documentation, smaller ecosystem, and several smaller usability issues.
That’s not to say that there are no disadvantages to Julia. I personally see Julia as a beefed up, new and improved R.
However, Julia is perfect if you write mathematical/physical/engineering simulations and data analysis codes, which is my typical use case. Its support for multiple dispatch and custom operators lets you to write very efficient code without sacrificing readability, which is a big plus. Support for HPC computing is very good too.
That's the case now, because Julia made a design decision to focus on extreme composability, dynamism, generic codegen etc which involved compiler tradeoffs...but it's not inherent to the langauge.
For scripting, interpreted Julia is coming. For executables, small binary compilation is as well...particularly bullish on this given the new funding
That's a bit of an understatement. It's about as fast as C and Rust (ignoring JIT compilation time).
designers seems to have a good amount of PLT knowledge and made good foundations
Well, I still might use Julia personally in the future, but my days of contributing to the community are over.
The julia community is thankless, and the people at Julia computing are mean spirited and they think it's acceptable to harm people having good intentions (like me), having contributed thousands of hours of free time to help people on their forum and releasing my own packages used by thousands of people.
Julia Computing just raised a bunch of money, but they are just pocketing it for themselves while they stomp their foot on people they perceive as unworthy.
Yea, it's a good language, but the people behind it are very selfish and harmful to the free software community.
