undefined | Better HN

0 pointsmjw_byrne1y ago0 comments

If you disallow control characters so that you can use them as delimiters, then CSV itself becomes a "binary" data format - or to put it another way, you lose the ability to nest CSV.

It isn't good enough to say "but people don't/won't/shouldn't do that", because it will just happen regardless. I've seen nested CSV in real-life data.

Compare to the zero-terminated strings used by C, one legacy of which is that PostgreSQL doesn't quite support UTF-8 properly, because it can't handle a 0 byte in a string, because 0 is "special" in C.

0 comments

saulpw1y ago

Nested CSVs as you've seen in real-life data are a good counterexample, thanks for providing it.

thayne1y ago

So have a way to escape those control characters.

mjw_byrneOP0y ago

Right, but the original point I was responding to is that control characters are disallowed in the data and therefore don't need to be escaped. If you're going to have an escaping mechanism then you can use "normal" characters like comma as delimiters, which is better because they can be read and written normally.

thayne0y ago

But a comma is much more likely to need to be escaped.

1 more reply

j / k navigate · click thread line to collapse