Erlang/OTP 25.0 Release (opens in new tab)

Previously (Erlang/OTP 24 highlight)

"The BeamAsm JIT-compiler has been added to Erlang/OTP and will give a significant performance boost for many applications. The JIT-compiler is enabled by default on most x86 64-bit platforms that have a C++ compiler that can compile C++17."

Easily the best feature in this release if you do your dev work on an Apple Silicon Mac (or Raspberry Pi, I guess?).

Also great if you have the option of deploying on AWS Graviton or similar.

This is great. I hope this finds its way to Elixir land.

IMO people represent what they know.

People also overestimate what “the new” is, constantly. Otherwise, PHP would be dead by now rather than flourishing.

Serverless is essentially PHP in CGI mode, with a lot of marketing behind it. There are certainly places where it’s beneficial.

Servers and databases are not going away anytime soon. They are simply too effective for what they do.

The “everything is legacy crowd” hasn’t been correct about almost anything in my lifetime. Mainframes are still going strong. SQL databases are better than ever and the NoSQL crowd has largely died out.

90% of the cloud ecosystems of today are just doing what Java app servers were doing in the late 90s.

Erlang and the BEAM offer a set of trade offs that happen to be ideal for a lot of server based use cases to make hard things really simple. Probably not going to take over the world but if it fits what you do you probably won’t want to go back.

My thinking is Erlang’s been powering massive systems since the early 90s, I’m feeling pretty good about its longevity.

Many of Elixir’s strengths are Erlang’s strengths.

I wouldn’t hesitate to learn some Elixir if it’s interesting to you. It’s not as pervasive as other languages but it’s not niche either.

Full disclosure, it’s been such a wild success for my team at New Relic the last 5 years that I’m very biased :)

> I've seen more than once recently an engineer saying that software solutions outside of the Serverless ecosystem (FaaS, DB, etc) is "the new legacy".

There's, as I see it, three different kinds of legacy: (1) code that has an inordinately costs to maintain and especially update to changing requirements because it lacks tests, documentation of existing business intent and/or design, or otherwise has lost institutional knowledge needed for nontrivial work, relying on the foggy memories of a high priesthood increasingly, over time, working through ritual without understanding, (2) software with deep and fundamental dependencies on platforms or external components that are no longer supported or no longer available, and (3) software which doesn't meet the current arbitrary tech platform decisions of the organization using it, but which has no fundamental problem preventing maintenance or use.

In lots of places, software that isn't adapted to a particular serverless architecture that the org has adopted is now the third type of legacy.

> And that cloud providers have solved all the pains that Erlang was supposed to address.

They haven't, though for some use cases, the parts of the stack Erlang would be most useful for implementing are the parts you are likely to just get from a cloud vendor (that's not true of all use cases, though.)

> Elixir sounds very compelling to me, but I worry that I might be going in a direction that's not where the industry is going.

Elixir is probably not the best bet if your concern is to direct the most focussed possible effort learning the things that are most likely to dominate the largest share ofnthr general software dev marketplace in the near future.

Serverless systems in almost every implementation are still stateless or limited stateful-serverless (the best examples are probably Microsoft's Durable Functions or Flink's Stateful Functions - there's also solid attempts by the Akka folks).

These are largely built on top of either a stream processing (in case of Flink) or some flavor of Actor model based on event sourcing for peristence.

What I'm getting at is that you'd build a serverless system on top of Elixir/Erlang/OTP and be served well by its mature actor model implementation and excellent distribution options. See https://eigr.io/ .

The cloud provided options are interesting for some experimental / tactical use cases but I don't yet trust their abstractions to not be leaky and they typically come with vendor lock-in trade offs.

I'm not sure where the industry is going but the innovations around LiveView are really convincing (I'm using LiveView in my work).

Some great innovating Elixir projects:

Oban: https://getoban.pro/

Oban Demo: https://getoban.pro/oban

Livebook: https://livebook.dev/

> What's your feeling on this?

That anybody claiming serverless has "solved all the pain" is definitely trying to sell me something.

I'd actually characterize it the other way around: Elixir and Beam give you most of the benefits of serverless with better portability and lower costs.

Also, meta: don't worry too much about where people tell you the industry is heading. 1) They don't actually know, 2) it's much more conservative out there than you might think. Even when new paradigms do gain traction, they pretty much always augment rather than replace, for a good long while at least.

If you're wanting to use Elixir, it's not going to be nearly as mainstream as, say, Node, but fly.io has support for Elixir apps (to speak of cloud), and I think there's some cool stuff going on in Elixir myself.

Of other note is that Erlang seems to enable folks that use it to punch above their weight class when it comes to running services internally and such.

Serverless is far from the only valid way to run things these days.

While Serverless does address some of the points which were the design goals of Erlang, it doesn't address them all. In no particular order:

Hardware interaction is more or less non-existent in the world of Serverless.

Timing/Latency is not really controllable. It's at the whim of your provider.

FaaS doesn't support very large systems well. It's far more about scaling down than up.

Erlang is functional for a reason: it minimizes faults. Quality follows.

Complex feature interaction is limited in the stateless serverless world.

Cloud providers tend to solve for a different set of problems, which is fine. But I'm going to claim that many of the things Erlang was designed to solve runs dual to the problems in the cloud space.

I will bet real money Erlang and elixir will be around and doing just fine in a decade. I would not make that same bet about any cloud services provider.

Serverless works super well for getting off the ground but the goal is to grow big enough that it’s no longer the best option, no?

Just because the industry is willing to waste money on tech they’ve outgrown doesn’t mean you need to.

One current advantage: FaaS is sold below cost by every cloud provider at the moment, so if you can design your app to take advantage of it, you'll save $$$.

Don't know how that'll hold up into the very long term though...

One good business cycle in the XaaS spaces might disabuse those people of that notion. Even on more stable providers, the whateveraaS products are a good solution for some cases, but far from panacea.

Serverless are rebranded CGI scripts for a new generation that buys into whatever developer advocates want to sell them this week.

There is a huge community, not just in Erlang, but in the larger BEAM ecosystem. Have you tried the Erlanger or Elixir slack? We also have https://erlangforums.com/ and https://elixirforum.com/. For smaller BEAM languages like Gleam, we have Discord available https://discord.com/invite/Fm8Pwmy

There's a lot of great stuff in the BEAM ecosystem, I would caution on writing it off too fast

OTP is different, but there are a lot of good resources for understanding it. I liked:

Elixir in Action.

The Little OTP Book.

Designing Elixir Systems with OTP.

Pragmatic Studio's Elixir OTP course.

I mean we already got a stable 25% or more improvement, to the point that Jason is faster than NIF based json library... we already saw massive perf improvement with the JIT

The new `maybe ... end` looks nice.

>engineer saying that software solutions outside of the Serverless ecosystem (FaaS, DB, etc) is "the new legacy"

It's complete blindness to the reality. Trusting one single provider for your whole service (often not even completely opensource) is the same trap as Windows or Oracle in the 90's...just worse (because not on prem)...but hey history repeats itself ;)

Serverless? You mean vendor lock-in, distribution issues and reproducibility issues, etc?

> cloud providers have solved all the pains that Erlang was supposed to address.

Assuming this is talking about tools like Kubernetes, they operate on a different level. The OTP's failure tolerance model operates on a (relatively) very small scale inside your application, whereas Kube operates on a whole-application level. Rather than obsoleting OTP, Kubernetes actually complements it.

There are binaries and strings. Binaries is what are used as efficient "strings" most often these days. Binaries allow matching, even building bitstrings (binary strings not ending on an 8 bit boundary). Even more interesting, strings and binaries can be combined in iolists, which is an arbitrarily nested or combined list of binaries or other lists. Those can be used for efficient IO and IO drivers know how to emit or consume those.

For additional details I recommend https://adoptingerlang.org/docs/development/hard_to_get_righ... which also mentions how unicode is handled. It turned out quite a bit easier because of how Erlang represents strings.

BEAM is never going to win for straight line speed. This is going to be an improvement for a language optimized for resilience, concurrency and consistency.

Running millions of concurrent processes, with their own heap, with consistent response times that individually tolerate failure without negatively affecting the entire system is just a different animal with a different set of trade offs.

You’re trading some speed for those benefits.

If you are allowed to write e.g., a Java program and run it on the JVM, the speedup will typically be on the orders of magnitude in favor of the JVM. Static type systems and extensive optimization matter.

However, chances are you have an Erlang program, and it's quite large. In this case, the speedup compared to the older bytecode interpreter is good for many programs. It is performance you get "for free" by upgrading.

The underlying reason for picking something like Erlang is robustness. Java isn't really built for this kind of programming, and the JVM doesn't directly facilitate it either.

I don't think the two JITs really do the same work, so it's hard to benchmark them against each other.

Erlang JIT takes BEAM bytecode and turns it into native code once at load time in order to avoid the overhead of the threaded code interpreter; if Erlang JIT is enabled, the interpreter is not used at all. Java's HotSpot JIT takes JVM bytecode and turns it into native code at runtime in order to improve performance of code that is run frequently (thus the name), guided by runtime information; lightly used code will be interpreted, heavily used code will hopefully be JITed to native. When HotSpot JITs, it may apply assumptions based on runtime use to produce faster code and if the assumptions don't hold for a call, the interpreter can be used instead (and if that happens often enough, the native code can be discarded and perhaps rebuilt), so the interpreter needs to be present always.

Erlang JIT has been focused more on always applying and speed of application (because it delays code loading), whereas HotSpot focuses more on getting performance gains when it applies, and speed of application is less important because it happens asynchronously. So if you benchmarked speed of applying the JIT, Erlang would probably win, and if you benchmarked optimization of the code, HotSpot would probably win, but either way you're comparing apples and oranges.

The JVM is nearly as fast as hand optimised C, this is not intended to be that fast, rather a simple bedrock to build further optimisations into. Immutable everything also will limit performance as will the focus on latency at the cost of everything else. Not sure of actual performance compared to JVM though, I wonder if it’s as far away as I expect…

Erlang JIT is still very simple, unlike highly optimized super complex V8 jit. It's faster than non-jit erlang, sometimes twice as fast, but nowhere near nodejs-fast. Erlang team don't have resources to make and maintain such complex beast. (everything said here is about raw performance)

Here's a good comparison between Elixir, Go, and Node:

https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixi...

Some of the things I like better about Elixir and Erlang vs. Go and Java are:

- In Elixir/Erlang, all processes are stored in private memory and can only be accessed via defined message interfaces. In both Go and Java, processes are stored in public memory.

- Elixir/Erlang processes are a lot smaller, memory-wise, than Go/Java processes.

- We moved all of our development at my current company from Java to Elixir because Elixir, being functional, makes writing multi-threaded applications a lot faster/easier. Elixir is also cheaper to deploy. At least that has been our experience.

- I like the syntax of both Elixir and Erlang better than Go, Java, or JS.

- I don't have as much experience with Node/Express, but in my limited testing with Node/Express vs. Elixir/Phoenix, the latter is several times faster and much more scalable. It also makes better use of hardware (which I believe is discussed in the link above).

I hope at least some of this helps answer your question.

fwiw https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

fyi "1.4 What sort of problems is Erlang not particularly suitable for?"

https://www.erlang.org/faq/introduction.html#idp32114576

I think the focus has been more on comparing the JIT'd code vs the unJIT'd previous VM approach and looking at the speedups.

In one sense, I totally understand your point. In another, I still got the message so who cares?

There are some really high integration ARM servers, like Cavium Thunderheads. And others. They're all unobtanium.

Could be repurposed for Android development. I don't see it happening soon. Binding to Erlang is pretty difficult, see https://github.com/joshnuss/elixir-ffi.

If it did happen it would be useful, just... take some time.

** Warning: Turns out this is a really bad idea, do not use! When the BEAM calls native functions it needs to know how many reductions should be used. Otherwise calling a long running function can break the scheduling completly, because with this approach a long running function is considered 1 reduction **

I haven't used it, but in a recent Thinking Elixir episode, they talked about Elixir-Desktop, which supports Android and iOS, as well as the standard Windows/Mac/Linux

https://github.com/elixir-desktop/desktop

I don't see any serious attempt at UIs with erlang. I think erlang processes would provide a great foundation for UI components, but has it even been tried?

Probably mostly focused towards running on Graviton et al.

Not yet, you can use OTP23 or OTP24 at the moment with latest Elixir 1.13. Check https://hexdocs.pm/elixir/1.13/compatibility-and-deprecation...

Yes. I'm currently using OTP 25.0 and Elixir 1.13.4 and it is working fine.

Damn thats funny! Thanks for sharing.

“Go home and take my nine nines with me.” Hilarious! Thank you.