The obvious thing would be making sure that it's easy to get a unambiguous, natural representation of what a float value actually is. (Similar to what C printf %a specifiers produce, rather than something like Ryu float-to-decimal, which is useful for what a float value means for human-readable purposes.)
0X1.999999999999AP-4
+0X1.999999999999AP-3
=0X1.3333333333334P-2
≠0X1.3333333333333P-2
is ugly (and could be improved[0]), but far more understandable (and less superstition-inspiring) than: 0.1 + 0.2 = 0.30000000000000004 ≠ 0.3
0: At the very least, please make it easy to get "0xA.BCDEFp+1", with capital "ABCDEF" and lowercase "xp", so the digits look like digits and the punctuation looks like punctuation.