undefined | Better HN

0 pointswesturner4mo ago0 comments

If you don't handle NaN values, and there are NaNs in the real observations made for example with real sensors that sometimes return NaN and outliers, then the sort order there is indeterminate regardless of whether NaN==NaN; the identity function collides because there isn't enough entropy for there to be partial ordering or total ordering if multiple records have the same key value of NaN.

How should an algorithm specify that it should sort by insertion order instead of memory address order if the sort key is NaN for multiple records?

That's the default in SQL Relational Algebra IIRC?

0 comments

thomasmg4mo ago

> then the sort order there is indeterminate

Well each programming language has a "sort" method that sorts arrays. Should this method throw an exception in case of NaN? I think the NaN rules were the wrong decision. Because of these rules, everywhere there are floating point numbers, the libraries have to have special code for NaN, even if they don't care about NaN. Otherwise there might be ugly bugs, like sorting running into endless loops, data loss, etc. But well, it can't be changed now.

The best description of the decision is probably [1], where Stephen Canon (former member of the IEEE-754 committee if I understand correctly) explains the reasoning.

[1] https://stackoverflow.com/questions/1565164/what-is-the-rati...

westurnerOP4mo ago

Were the procedures for handling Null and Null pointers well defined even for C in 1985 when IEEE-754 was standardized?

There's probably no good way to standardize how to fill when values are null or nan. How else could this be solved without adding special cases for NaN?

In a language with type annotations we indicate whether a type is Optional:

  def sum(a: float|None, b: Optional[float]) -> None|float :

  def sum(a: float|np.nan|None, b: Optional[float|np.nan]) -> None|float|np.nan :

westurnerOP4mo ago

What is a good sort key for Photons and Phonons? What is a good sort key for H2O water molecules?

j / k navigate · click thread line to collapse

0 pointswesturner4mo ago0 comments

How should an algorithm specify that it should sort by insertion order instead of memory address order if the sort key is NaN for multiple records?

That's the default in SQL Relational Algebra IIRC?

0 comments

thomasmg4mo ago

> then the sort order there is indeterminate

The best description of the decision is probably [1], where Stephen Canon (former member of the IEEE-754 committee if I understand correctly) explains the reasoning.

[1] https://stackoverflow.com/questions/1565164/what-is-the-rati...

westurnerOP4mo ago

Were the procedures for handling Null and Null pointers well defined even for C in 1985 when IEEE-754 was standardized?

There's probably no good way to standardize how to fill when values are null or nan. How else could this be solved without adding special cases for NaN?

In a language with type annotations we indicate whether a type is Optional:

  def sum(a: float|None, b: Optional[float]) -> None|float :

  def sum(a: float|np.nan|None, b: Optional[float|np.nan]) -> None|float|np.nan :

westurnerOP4mo ago

What is a good sort key for Photons and Phonons? What is a good sort key for H2O water molecules?

j / k navigate · click thread line to collapse