I thought the benefit/raison d'etre to Clojure is the JVM ecosystem.
Yep, they learn Ruby and then they need to go JRuby when performance becomes relevant. :)
The real benefit of native Clojure to me would be for reducing startup time and the JVM overhead in containers. The best option for this right now is CLJS, but it would be nice to just not have a runtime.
still i agree with you, having native clojure would be amazing. there is pixie which is transpiled by rpython into c++ but it's pretty much abandoned.
* https://github.com/tptodorov/aws-lambda-scalajs
* http://underscore.io/blog/posts/2016/03/21/serverless-scale-...
I am excited that now there will be an option for getting both quick startup and fastest execution time!
Do you mean it might convince Amazon to add Scala as a supported language? You don't get access to run native code on AWS, just Python, Java, C#, or JS.
And, even with a robust, mature native implementation, I'm not sure that Scala would be the top priority for the next language.
etc. etc.
Scala.js lets you run Scala code on javascript-based VM's and provides full integration with the underlying platform and libraries. Scala native looks to be another attempt to expand Scala's reach into new platforms beyond the JVM.
What are the alternatives? I couldn't live without HKT these days (once you're used to thinking in them it's painful to work without them), which rules out most languages even in the ML space, and Scala has better IDE support than anything that's left (indeed an excellent tool ecosystem in general in terms of e.g. profilers, instrumentation) except possibly Ceylon (which doesn't have anything like the library/developer ecosystem of Scala). Even assuming Ocaml lands their modular implicits functionality doesn't change this as far as I can see.
Haskell is an option, but eager evaluation makes reasoning about performance much easier. (And I do think there are legitimate use cases for traditional OO inheritance, though this is debatable). If and when Idris reaches a similar level of library/tool support to Scala then it might become a better alternative to Scala Native, but not that many people are willing to use Idris in production yet.
To me, Scala's biggest benefits are just the language itself. So many little things that I couldn't do in Java, I can do with little effort in Scala. I can write imperative code if I really want, but I have all the benefits of functional programming available too.
While the VM in LLVM historically was short for Virtual Machine, it really has nothing to do with that. It's a compiler backend used by Clang (C++ compiler) and Rust.
Did you test scala native against well known and massive open source scala project ? Did the performance improved or regress ? Did you wrote a brand new scala compiler for native code ?
I disagree somewhat with this.
We found that when we started writing microservices in languages that are not java, the short startup time changed how we did some error handling.
For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic.
The problem with reconnect-logic is that it is code that (may) be tested very rarely. This in turn means it's easy to get strange long term problems there like very slow memory leak due to some listener being added to a connection object once the connection is initiated.
We did a 180 on reconnect-logic in our nodejs-processes and let the exceptions just bubble unhandled and take the entire vm down. With automatic restart script, the process will be back in seconds anyway, and with docker having built in back-off timers for auto restart, we don't necessarily overload the shared resources.
[1] http://www.scala-native.org/en/latest/user/lang.html#lang
With microservices run as containers that are started on demand it matters; with other architectural choices it may matter less.
- faster startup time
- (drastically) lower memory footprint
- fine hand-tuning of you application
- Predictable latency if desired (optional GC)
- Very low call overhead for C ABI
The following comparison is also interesting, with a JSON serialization example in Swift outpacing Spring/Java by a factor of ten... This is also running on Linux instead of macOS.
https://medium.com/@qutheory/server-side-swift-vs-the-other-...
1) Much faster startup times
2) smaller memory footprint for small programs.
3) Potential for easier installation since no dependency on the JDK (assuming binaries are statically linked)
So basically you could use scala native to cover some cases that are better covered by golang or rust right now. For large and long-running server-side processes, the JVM is still king.
I also did not get on well with the community, which seemed to have a lot of people with the attitude - "they won't let me use haskell at work so I'll make do with this shit". They didn't seem to understand or be interested in OO at all, and were very fanatical about driving application logic with types, purity, and the like.
Regardless, a native variant would be something well worth investigating if it ever reaches "production ready".
I think a lot of people see a dense 1-liner and panic when they can't read it as fast as a line of another language, when actually the same operation would be 5 or 10 lines in that other language. It's ok to take a bit longer to read it; in my experience if you take a deep breath and read slowly, breaking it down into pieces, it's all very simple - the combination may be complex, but everything is very modular and compositional so you can understand each piece in isolation and then put them together to make the bigger picture.
> I distinctly recall writing a 100 line or so program where the type declarations crashed the compiler.
Do you still have it? Those are pretty rare (and almost always a type error rather than a correct type, though I appreciate that doesn't help you much at the time) and I'd be interested in minimizing it into an actual bug report.
> And the tools themselves were huge memory hogs
Yeah that does happen. I avoid SBT, but eclipse isn't exactly light. They're putting a lot more focus on performance in recent releases which will hopefully address this up to a point.
> I also did not get on well with the community, which seemed to have a lot of people with the attitude - "they won't let me use haskell at work so I'll make do with this shit". They didn't seem to understand or be interested in OO at all, and were very fanatical about driving application logic with types, purity, and the like.
The community is pretty awful. But I found that as people got more experienced, myself included, they tend more towards the pure, strongly typed side of things and make less and less use of OO. I understand OO but am no longer interested in it, and I think most experienced Scala folk feel the same way, so while the language does have good support for it, you'll struggle to find many people interested in helping you with it. I'm not sure anything can be done about this - you can't expect the experienced users to answer questions they're not interested in.
And it's not so much that I hate type driven programming or purity. I just view them as tools in my toolbox, not as methodologies or worse - religions. I still think objects are an amazing concept if you look at them through the smalltalk lens. Or if you take a moment to step back and realise that an anonymous function is very similar to an object with an "apply" method, or that a constructor for an immutable object is like a functor that partially applies many related functions at once.
On the other hand, I wonder why such an effort was never carried out with Java itself? Or maybe it was but just never took off?
I used Apache Harmony as the standard library, and binaries were about 2MB, which back in 2012 I considered unacceptably large; benchmarks were adequate only. I never got round to implementing a garbage collector but the hooks were in place for a simple mark/sweep stop-the-world collector. The hard part would be to actually make all threads of the program stop so they could be collected; I had a plan, but no implementation.
Interested parties may also be interested in my experimental Java JIT which worked by converting bytecode into Lua and then running it with LuaJIT:
It actually beat OpenJDK Server for some unrepresentative microbenchmarks. It's very unfinished, and generates fairly pathological Lua code which LuaJIT has trouble with, but the biggest problem is that Lua and LuaJIT can't do shared-memory threading, so most Java code won't work on it.
(JVMs are surprisingly easy to write. If you're interested in interpreters or code generation, I encourage you to try it.)
(Man, I completely forgot I wrote these until now!)
It didn't take off for two reasons.
Firstly, it's not really any faster than a good JIT, and Java has very good JITs. The startup time is better, but the startup time of a JVM itself is actually pretty small ('time java -cp . HelloWorld' prints 'real 0m0.112s' on my machine), and for the uses Java is usually put to, irrelevant. I'd be very interested to see some benchmarks for Scala Native.
Secondly, there's enough dynamic code in the Java ecosystem - bytecode weaving, dynamic proxy generation, plugin classloading, etc - that in practice, your AOT-compiled application need to have an interpreter along for the ride anyway. That means you don't even save the space overhead and distribution complexity of deploying a JVM.
The key advantage of Scala in general is that these things are rarely necessary - the fancy type system and generally expressive language make it much easier to address the use cases with "vanilla" Scala instead. So Scala Native will hopefully be able to bypass this issue.
The core JDK makes those assumptions and is indeed a bad candidate for AoT OTOH so is scala and it will prove very difficult to provide any value beyond what the JDK already offers.
Title says Kotlin, but it is about JVM languages going native, in general. Or should they?
The Java ecosystem is playing huge catch up here, and I don't think it's a bad thing that people are exploring alternatives, whether that's Avian, the now defunct RoboVM, or LLVM-based backends.
The only thing is that most ignore there is a JVM world outside OpenJDK.
And even those that know that world aren't willing to pay for commercial JVMs.
Thus propagating the myth that Java doesn't support AOT.
Which in a way is also understandable, given Sun's political decision that AOT compilation was tabu.
Luckily Oracle isn't Sun and has heard the industry. So if all goes as planned, by Java 10, those that don't want to pay for AOT compilers, will have one in OpenJDK.
I'm not a regular Scala user.
Quick question: does this compile down to DOT before going to LLVM? Or has DOT not yet arrived in Scala Native?
[1] http://www.scala-lang.org/blog/2016/02/03/essence-of-scala.h...
They say one of their targets is using this for command-line tools (I'm guessing for startup speed and needing to be small in memory footprint) but it's not of much value if an "echo" or "grep" implementation takes up 15 to 30MB on the drive.
In this day & age? It might be of value if an echo or grep takes ~20MB and a full-blown HTTP REST server with DB access takes ~25-30MB. Why do I care again exactly whether there is a fixed portion of bootstrap/runtime core code in the binary, as long as I'm not writing echo or grep?
(Any helloworld app not written in asm (or lib-less C) will consist largely of "non-helloworld code" in terms of "bytes occupied in the binary", right?)
Dunno might be of concern eg with embedded/raspberry and such, but for code to be "moved from a jvm-kinda flair to a go-kinda flair" I'm not seeing the issue .. yet ;)
In my eyes, the main value proposition of Scala native will be "you don't have to learn a new language if you need to write an echo or a grep", with "you don't need to learn a new language if you need to run some of your code on a platform that does not have a reasonably good JVM" a distant second. The former would suffer a lot from oversized binaries, the latter depending on the specifics of the target environment.
The goal was to create JVM suitable for standalone GUI applications. Project was abandoned when Sun/MS Java wars started in favor of the Sciter (https://sciter.com).
As of NativeScala ... I think that approach (binary with nano JVM + attached class files) may work better and with less effort. Scala needs JVM infrastructure, GC, etc. as far as I understand.
P.S. I think this is related to rust, in a sense before Rust there was no serious competitor to C/C++, but after seeing what Rust doing to C/C++ I think there will be more native language to compete with in low level area.
On the other hand, the new targets (js and native) might make Scala more interesting to Java pragmatists who don't care much about going functional but would not mind writing Java in a more streamlined syntax.
People do [1], and are very happy with it.
Not sure if I get this, don't Java VMs support this use case (J2ME) ?
It must be frustrating to work on a project like this, see areas where the language can be improved, but only be able to do the work to make it purely compatible. Hopefully some good comes out in the form of some good SIPs.
On the other hand, it's also worth bearing in mind that this is version 0.1.0; over time, some of these limitations will lift. What i don't know is whether Scala Native will develop into a complete version of Scala which compiles to native code, or evolve into a variant of Scala more tightly adapted to a niche of talking to native libraries.
Anyway ...
(1) No threading [1]:
Scala Native doesn’t yet provide libraries for parallel multi-threaded programming and assumes single-threaded execution by default. It’s possible to use C libraries to get access to multi-threading and synchronization primitives but this is not officially supported at the moment.
So forget about using Akka for now.
(2) NullPointerExceptions are replaced with segfaults (hopefully) [1]:
A number of error conditions which are well-defined on JVM are undefined behavior: Dereferencing null. Division by zero. Stack overflows. Those typically crash application with a segfault on the supported architectures.
That's not so bad; where Java apps might let nulls flow around and rely on catching NullPointerExceptions to recover from them, Scala apps are much more likely to use Optional consistently.
(3) If you do want to talk to a native library, and you need to allocate memory to do it, you're on your own [2]:
Unlike standard Scala objects that are managed automatically by the underlying runtime system, one has to manage native pointers manually.
Scala Native provides a built-in way to perform stack allocations of unmanaged memory using native.stackalloc function: [...] When using stack allocated memory one has to be careful not to capture this memory beyond the lifetime of the method. Dereferencing stack allocated memory after the method’s execution has completed is undefined behaviour.
Scala Native’s library contains a bindings for a subset of the standard libc functionality. This includes the trio of malloc, realloc and free functions
Java's traditional JNI is a verbose, slow, pain in the stdout, but it was designed pretty carefully to avoid problems like this.
(a) Intermission! Check out how they do type-level numbers [1]:
Natural numbers are types that are composed of base naturals Nat._0, ... Nat._9 and an additional Nat.Digit constructor.
That's a new one on me!
(4) Incomplete JDK libraries [3]:
Scala Native supports a subset of the JDK core libraries reimplemented in Scala. Here is the list of currently available classes: [...] This is an ongoing effort, some of the classes listed here might be partially implemented.
The list has most of the fundamental stuff - a good chunk of java.io and NIO, the collections, java.lang, atomics. But no java.text, java.net, concurrency, regexp, date and time, JDBC, reflection, XML, etc.
They don't mention how much of the Scala libraries they support. I would imagine that they can build anything that's in pure Scala and depends only on JDK classes in that list, so you'll get the core language stuff and the collections. Not sure.
[1] http://www.scala-native.org/en/latest/user/lang.html
I imagine building this stuff on Windows will be challenging :/
It is a problem that many compilers cannot be bootstrapped from source without a trusted binary of a previous release.
Is that such a big deal?
There are several compiler implementations that enable bootstrapping from alternative implementations, which shifts the problem to a simpler language, which may already have a bootstrap path.
is it a theoretical problem or a practical problem for industry users?