undefined | Better HN

0 pointscbzbc6mo ago0 comments

I'd like to know the details of the errors -- because it could have been as simple as running out of memory.

0 comments

I doubt this hypothesis, because duckdb written in c++ should be able to tolerate memory failure, while this written in rust has to deal with rusts memory allocation failures are panic's behavior.

That is to say that if the issue is duckdb running out of memory, it is most likely because the rust implementation is using memory more efficiently for whatever query is crashing duckdb, rather than graceful handling of memory allocation failure.

Where it is possible in c++ to gracefully handle memory allocation failure, it is not really a thing in rust I'm not even sure whether it is possible to catch_unwind it. I say this as a rust person who doesn't fancy c++ in the slightest...

deschutes6mo ago

You cannot use the rust standard library in environments where arbitrary allocations may fail but neither can you use the STL. The difference is the rust standard library doesn't pretend that it has some reasonable way to deal with allocation failure. std::bad_alloc is mainly a parlor trick used to manufacture the idea that copy and move fallibility are reasonable things.

I wouldn't wager a nickel on someone's life if it depended on embedded STL usage.

vlovich1236mo ago

I’ve never seen anyone try to catch allocation failures in C++ code and in many cases doing so correctly is very difficult, not least of which is that writing exception-safe code is the exception, not the rule.

eptcyka6mo ago

There's an effort to expose allocation errors in the standard library for the Linux kernel. Pretty sure it is well under way.

cyanydeez6mo ago

OOM are still something a DB can "avoid" so it's not like that class of bugs is some special issue that nullifies thing.

cinntaile6mo ago

Crashing when running out of memory is not acceptable software behavior in my opinion.

cbzbcOP6mo ago

Right, but all it says is that an error was thrown.

MrPowers6mo ago

You can generate the dataset with the instructions in this readme: https://github.com/apache/sedona-spatialbench/tree/main

Here are the queries: https://github.com/apache/sedona-spatialbench/blob/main/prin...

They should be fairly easy to replicate!

j / k navigate · click thread line to collapse