undefined | Better HN

0 pointsprodigal_erik13y ago0 comments

Unicode 2.0 added surrogate pairs in 1996. Unfortunately, the first versions of both Java and JavaScript predated this and got strings horribly wrong, and now any conforming implementation of either is required to suck. The Right Thing would be for almost everyone to work with only combining character sequences, except for a rare few who need to know how to dissect one into its codepoints and reassemble them correctly (just as people don't normally need to extract high or low bits from an ASCII character).

0 comments

jrabone13y ago

No. Combining characters and NF(K)C/D normalisation rules are a different problem entirely - consider the "heavy metal umlaut" (ie. Spın̈al Tap) where there is no lossless conversion possible - only “n" followed by U+0308

prodigal_erikOP13y ago

They're facets of the same problem. I shouldn't routinely be dealing with either surrogates or combining marks; unless I have a specific reason, it's only an opportunity to make a mistake that hardly anyone knows how to troubleshoot. "n̈" should be an indivisible string of length one until I need to ask how it would actually be encoded in UTF-16 or whatever.

jrabone13y ago

But that's the point - there is no such character. Given the Unicode consortium have added codepoints for every other bloody thing under the sun, I'm amazed that there isn't one for n-diaresis but there you are.

Add a small number of people who for artistic reasons decide that they want to make life hard (Rinôçérôse I'm looking at you) and you just have to accept that the length of your string might not equal the number of codepoints contained therein...

j / k navigate · click thread line to collapse

0 pointsprodigal_erik13y ago0 comments

0 comments

jrabone13y ago

prodigal_erikOP13y ago

jrabone13y ago

j / k navigate · click thread line to collapse