Erlang/OTP 24 highlights (opens in new tab)

Recently I read somewhere that writing long prose is easy but writing something succinct takes way longer. I think that translates to writing code as well.

Imagine if we had a metric of "how easy is the code to understand and work with" rather than "can we do it with less lines of code".

> So a net reduction of 60K lines of code? And yet functionality was added to the system as well. That's praiseworthy IMO.

Yup.

> Imagine if "we did more with less" infected much of software development today.

Taken to its logical conclusion this should mean that the ultimate piece of software, that does everything and does it perfectly, is 0 LoC.

Hmm, fells like I need to check my logic here...

I just ran programs from computer language benchmarks with OTP24. Every program ran considerably faster almost 30% speed up.

I was hoping our `mix test` would be faster, but it doesn't appear to be. It would be nice if this got some attention fro the Elixir team.

It's noteworthy that the JIT doesn't (yet?) do runtime optimization or specialization, so gains should be moderate. The very low end of double digits.

Not comparable to going from a Javascript interpreter to v8.

But it's a great starting point.

The column number for errors and warnings is a welcome, long overdue addition.

Awesome, thanks for linking that! I was wondering how these improvements would flow through to Elixir.

Looks like we get the JIT performance improvements for free.

And unrelated to the Erlang/OTP changes, Elixir 1.12 looks awesome. Totally small, but such an unexpected little quality of life improvement, Kernel.then/2 for pipelines, looks great. I love the core team's focus on the UX of the language.

Do consider Elixir for its metaprogramming (macros) and the goodies that come with them, namely fantastic libraries and frameworks like Ecto and Phoenix.

Okay, lots of people arguing about this, so I'm gonna reply to you, the top-most person in this sub-thread :) I think the reason that it's easy to argue is that you can be talking about similar but slightly different things.

When Rust was created doesn't really matter, exactly. For a very long time, errors looked something like this:

  hello.rs:2:4: 2:16 error: unresolved name: print_with_unicorns
  hello.rs:2     print_with_unicorns("hello?");
                 ^~~~~~~~~~~~~~~~~~~

At some point, "improving the error messages" became a project goal, and Jonathan Turner decided to take this on. This is described in https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-com... . The post explicitly cites Elm as inspiration because Rust was directly inspired by Elm in this regard.

Yes, other languages may have started this trend earlier. Yes, maybe they influenced Elm, which influenced Rust. Yes, Rust may be "older" than Elm. But if you ask the people who began this effort, they will name Elm as the inspiration.

(And yeah, then you can try to argue about who has influenced the broader public, which is effectively impossible to prove, IMHO.)

Today that error looks like

  error[E0425]: cannot find function `print_with_unicorns` in this scope
   --> src/main.rs:2:5
    |
  2 |     print_with_unicorns("hello?");
    |     ^^^^^^^^^^^^^^^^^^^ not found in this scope

incidentally, and there's also JSON output, and if you're not on a terminal, you get the line numbers like before... lots of things are improved. This particular error doesn't show off some of the nicer things. Esteban Küber has taken up where Jonathan left off, and has been doing amazing work.

expressive diagnostics was one of clang's early selling points: https://clang.llvm.org/diagnostics.html

Clang was released 2007 and was usable 2009/2010-ish. Rust dev started 2010.

I'm not saying the trend of having good diagnostics was started by clang, but it's a more believable than the claim that it was started by Rust.

--

Rust-the-language is nice, but the Rust community feeling the need to mention Rust on every unrelated thread is a bit of a turn-off for me.

> Rust seems to have started that trend,

No it didn't. Lets not try to rewrite history here, clang started the trend of meaningful error messages, gcc quickly caught up.

Rust was inspired by Elm, if I’m not much mistaken.

Part of the problem here is that dialyzer is a quite ... spaghetti codebase, that noone really want to fund that work and that noone really knows how this stuff work that is interested in doing that work.

There is a lot of research work to do and noone to pay for it. That said, if you know of a company interested to fund this work, i am interested and would love to hear more about it.

Years here

Erlang already comes with a distributed DBMS called Mnesia https://erlang.org/doc/man/mnesia.html , which under the hood uses ETS/DETS depending on the configuration. In distributed Erlang projects you'll find that or abstractions over it. Mne sia makes the most sense in mu opinion when you only have 1-2 relations or just need some kind of distributed cache.

In Elixir apps you'll frequently find the aforementioned Ecto "ORM", which has adapters to different DBMSs like MySQL, PostgreSQL, and even Mnesia.

Postgres? ;) In all seriousness though, I wouldn't really expect database needs for Erlang & Friends to be that different from other languages. My current employer has a vertically scaled AWS RDS Postgres as companion for an Elixir app and it's worked great. (Ecto in particular is an excellent ORM.) If your app truly needs zero downtime or ultra-low latency then there are other options.

I've had interesting exchanges with a guy who does work on CouchDB. That is built with Erlang and should scale quite well in what sounds like a resilient way. Haven't dug in.

Horizontal scaling of the DB seems like a problem in many architectures. For SQL I'd be looking at CockroachDB as it is Postgres-compatible and is built with scaling out in mind. Haven't tried it though. Most of my work hasn't needed that for the DB recently.

Probably CockroachDB (https://www.cockroachlabs.com/)? It's wire-compatible with postgres so any library with a postgres driver will work.

Elixir has the excellent Ecto package which talks to many SQL dbs, Postgres being the default.

fwiw elapsed seconds

    Erlang/OTP 23 [erts-11.1] [hipe]
    Erlang/OTP 24 [erts-12.0] [jit]

    binarytrees,hipe,1,7.531
    binarytrees,erlang,1,11.768

    binarytrees,hipe,2,4.172
    binarytrees,erlang,2,5.149

    fannkuchredux,hipe,1,59.079
    fannkuchredux,erlang,1,73.151

    fasta,hipe,1,57.006
    fasta,erlang,1,50.843

    fasta,erlang,2,20.209

    knucleotide,hipe,1,92.662
    knucleotide,erlang,1,80.360

    knucleotide,hipe,3,86.793
    knucleotide,erlang,3,70.949

    mandelbrot,hipe,1,118.623
    mandelbrot,erlang,1,48.756

    mandelbrot,hipe,2,101.211
    mandelbrot,erlang,2,46.154

    mandelbrot,hipe,3,87.793
    mandelbrot,erlang,3,44.633

    mandelbrot,hipe,4,84.878
    mandelbrot,erlang,4,44.976

    nbody,hipe,3,140.025
    nbody,erlang,3,100.630

    pidigits,hipe,1,8.791
    pidigits,erlang,1,8.008

    pidigits,hipe,2,8.515
    pidigits,erlang,2,8.673

    pidigits,hipe,3,7.935
    pidigits,erlang,3,7.748

    regexredux,hipe,6,42.757
    regexredux,erlang,40.402

    revcomp,hipe,1,25.622
    revcomp,erlang,1,23.070

    revcomp,hipe,3,188.990
    revcomp,erlang,3,155.024

    revcomp,hipe,4,122.507
    revcomp,erlang,4,106.456

    spectralnorm,hipe,1,92.347
    spectralnorm,erlang,1,62.519

    spectralnorm,hipe,2,11.176
    spectralnorm,erlang,2,11.460
    

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

HiPE support is getting wonky, I don't know if many people use it, and I'm not sure it supports the latest OTP versions.

The Erlang folks are looking for maintainer volunteers to continue working on HiPE support, they don't have the manpower right now to maintain it themselves.

I'm not sure if there's a point in the history where you can run with (this) JIT or with HiPE on the same commit. Which makes an apples to apples comparison difficult. If you compare HiPE on OTP 23 with JIT on OTP 24, you're also getting the large amount of other changes as well.

Both HiPE and this JIT have drastically different improvements depending on the specific code that's running, which makes it challenging to have a real world benchmark as well.

There's a comment here: https://blog.erlang.org/the-road-to-the-jit/#maturing-the-ne...

I feel like I saw graphs in one of the presentations, but I don't recall which, or if it was about the final iteration of the JIT.

I suspect HiPE would beat the JIT on tight loops, but the JIT wins in general because of the lack of switching cost between native and interpreted code.

I believe HiPE is going away in the future.

I know you're being sarcastic but I will definitely not miss those when working with Elixir......

Seriously. I'm forever forgetting that Erlang functions called from Elixir often accept charlists as args.

Found the person who likes writing an in-house ETS wrapper on every new job!