[edit: apparently the author is a deno employee but did not choose to disclose such? If so then that throws any moral position they have out the window. The below however is still accurate to my reading of the complaints.]
First up is the FFI test. On the one hand I think that this is a legitimate complaint, changing an existing benchmark for your comparisons is always suspect. That said I can see technically legitimate reasons for the change - for example if bun's ffi interface doesn't actually handle byte arrays as well as other runtimes. But in such a case that should be explicitly called out as a caveat.
The sqlite complaint is not reasonable, if sqlite is being used as a demonstration of wasm performance there is nothing at all wrong with this test - if anything using wasm makes it more valuable as a comparison of runtime perf, otherwise you're very quickly going to just be benchmarking one native build of sqlite to another native build of sqlite.
The React complaint - this just looks like bog standard formatted minified code, which is an absolutely reasonable thing to do - in fact I would argue it is a much better reflection of real world performance than using the non-minified code. Production websites only ever send minified content (at the react scale) and minification absolutely impacts page load performance, and historically could result in catastrophic runtime impact (though they're better at not doing this now).
It seems like the author disclosed this a couple minutes after you wrote your comment.
> The sqlite complaint is not reasonable, if sqlite is being used as a demonstration of wasm performance there is nothing at all wrong with this test
Looking at that benchmark, it seems to be comparing Deno's WASM sqlite to Bun's native bun:sqlite, which I'd consider to be an unfair comparison.
Ah, then I have misunderstood what it's doing - that's hilariously terrible benchmarking, and if so the bun folk should feel bad.
Overall though, Bun is an amazing accomplishment for many reasons and I'd love if the project focused a bit less on the supposedly benchmark/performance and more in some other things it brings to the table, like it being based on `JavaScriptCore`, being an alternative to Node but still mostly compatible, the speed of development (vs Node which we've been complaining that has stagnated for years), etc. Not saying not to mention impressive benchmarks, just that it'd be nice to highlight the other points as well!
I personally also don't like the direction Deno was taking, and def welcome Bun as a modern alternative to Node.
Yeah, the story is:
Deno: Hi, we are a new nodejs alternative without all the mistakes nodejs made.
Bun: Hi
Deno: We are going to adopt all nodejs mistakes.
Anyone have more in-depth understanding?
Seems to be the case IMO, but it's not 100% cut and dry. Bun.js' position could be adequately explained in every case as "the defaults/obvious choices for these benchmarks in Deno are slow", but then again they are consciously making apples to oranges comparisons which feels dishonest to me.
(Also, disclosure, not disclaimer.)
I want to be clear, I am not saying that the post is wrong, just that we would know to be more careful in reviewing claims.
This is almost a meme in the Rust subreddit at this point. So many times, people come, post some kind of benchmark claiming that Rust is slower than X. Then after some back-and-forth discussion/tests, we found out that the release mode wasn't used.
(though medium seems to resist archive.org by reloading the page indefinitely)
I don't know if the suggested FFI benchmark is fair, as the change would no longer be simulating doing many ffi_hashes on many byteArrays, instead it simulates doing many ffi_hashes on the same byteArray over and over and benefits from caching the type casting necessary by deno.
Replace the const byte array with a randomly generated list (or real bytes you'd want to hash) of many passed to the benchmark as a parameter and the benchmark and the deno one loses the advantage from the cached type cast.
https://github.com/oven-sh/bun/commit/4b5c9acc72908ba22e3f70...
Then I go to the repo and see Bun is a Zig project.
I've maybe said too much already. :|
Another thing is I believe Deno supports WebGPU and Node supports WebGL via Angle etc (for e.g. running Tensorflow with GPU support)but Bun not only has no story there but its author has refused to provide any info about their intent aroundGPU support.
You mean just like libuv/node, the JVM and Go?
not sure I agree with the take, but that's how I read it
None of the issues have been fixed ofcourse. I don't think i care anymore.
This "exposé" seems like mud slinging from a disgruntled Deno maintainer with inherent bias to see Deno perform better. If anything, it paints a poor picture of Deno, not Bun.
Regarding the SQLite claim:
The "view source" link for SQLite is out of date (fixing shortly), but the numbers are correct. I forgot to change the label on the landing page from "x/sqlite" to "x/sqlite3" and I forgot to update the source link.
You can see I updated in this git commit:
https://github.com/oven-sh/bun/commit/4b5c9acc72908ba22e3f70...
Here is what it shows for me, but I encourage you to run it on your own computer to see for yourself:
deno run --unstable -A deno.js
cpu: Apple M1 Max
runtime: deno 1.26.1 (aarch64-apple-darwin)
benchmark time (avg) (min … max) p75 p99 p995
------------------------------------------------------------------- -----------------------------
SELECT \* FROM "Order" 26.3 ms/iter (25.06 ms … 29.23 ms) 26.74 ms 29.23 ms 29.23 ms
SELECT \* FROM "Product" 53.91 µs/iter (52.17 µs … 317.75 µs) 54 µs 65.63 µs 76.75 µs
SELECT \* FROM "OrderDetail" 269.41 ms/iter (240.72 ms … 308.82 ms) 279.05 ms 308.82 ms 308.82 ms
bun bun.js
[0.26ms] ".env"
cpu: Apple M1 Max
runtime: bun 0.2.0 (arm64-darwin)
benchmark time (avg) (min … max) p75 p99 p995
------------------------------------------------------------------- -----------------------------
SELECT \* FROM "Order" 14.44 ms/iter (13.71 ms … 17.82 ms) 14.56 ms 17.82 ms 17.82 ms
SELECT \* FROM "Product" 34.39 µs/iter (30.46 µs … 4.55 ms) 32.88 µs 48.5 µs 60.13 µs
SELECT \* FROM "OrderDetail" 148.17 ms/iter (144.8 ms … 154.92 ms) 149.85 ms 154.92 ms 154.92 ms
Regarding the FFI benchmark, you can see the commit from today here:https://github.com/oven-sh/bun/commit/40506e33e73018103bcf08...
It threw an error until I googled "Deno pointer ffi", which mentioned this function https://doc.deno.land/deno/unstable/~/Deno.UnsafePointer and that worked. I assumed that `Deno.UnsafePointer.of` is the expected way to get a pointer to a buffer, but it seems that changing the type to "buffer" is a faster way for this case. Will update the page shortly.
https://github.com/oven-sh/bun/issues/921
https://github.com/oven-sh/bun/pull/1056#pullrequestreview-1...
This makes SQLite transactions no longer serializable (in regard to the schema), and breaks the safety of any kind of external concurrency (e.g. mvSQLite and Litestream).
Unfortunately for supporters it may be the end of Deno (unless overly-aggressive core owners step out).
And now there's three: Deno, Bun and Cloudflare Workers
Is there anyway to unFlag it?