undefined | Better HN

0 pointsdrawkbox11y ago0 comments

You can still deconstruct the message and it is simple and easier to debug even if less exact.

In binary if there is one flaw the whole block is bunk, i.e. off by one, wrong offset, binary munging/encoding, other things. As an example if you have a game profile that is binary, it can be ruined by binary profiles and corruption on a bad save or transmission.

Binary is all or nothing, maybe that is what is needed for better standards but it is a major change.

What is easier, a json or yaml format of a file or a binary block you have to parse? What worked better HTML5 or XHTML (exactness over interoperability)

Granted most of us won't be writing HTTP/2 servers all day but it does expect more of the implementations to adhere to, for better or worse.

The classic rule in network interoperability is be conservative in what you send (exactness) and liberal in what you accept (expect others to not be as exact).

0 comments

MichaelGG11y ago

The "classic rule" aka Postel's Law, has proven to be disastrous. The idea of resuming a corrupted message is a totally flawed concept. At best, it introduces compatibility issues. This is essentially the history of HTML and browsers, each one needing to implement the same bugs as other popular versions.

SIP is another IETF gem, which takes its syntax from HTTP. And guess what? It's impossible to have unambiguous parsing in the wild! Why? The whole liberal in what you accept bad idea. So A interprets \n as a line ending, even though the spec says \r\n. B interprets it another liberal way, and assumes you didn't mean to transmit two newlines in a row, so it'll keep reading headers. End result: you can bypass policy restrictions by abusing this liberal-ness and get A to approve a message that B will interpret in another way. Yikes. And, since the software for both is so widely deployed, there is little hope of solving the problem. In fact, the IETF essentially requires you to implement AI as you're supposed to guess at the "intent" of a message.

So you're sorta proving my point, that people are thinking "oh it's just text" and then writing shitty, sloppy code, and they're giddy cause it sorta worked, even from a two line shell script. And then further generations have to deal with this mess, because these folks just can't bear to get line endings right or whatnot.

drawkboxOP11y ago

Keep in mind you are still going to have lots of these same problems you mention inside the binary blocks and header blocks. Just the specific annoyances of HTTP 1.1 will be gone but new ones will appear.

Going binary does not make it suddenly easier, it just slices it up and adds a layer of obfuscation.

Easier to know what the hell is going on across a wire with current formats and debug them. Utopia interop does not exist so Postel's Law has gotten us this far. Being text no doubt makes it easier to debug and interoperate, otherwise we'd be sending binary blocks instead of json. Unless you control both endpoints, Postel's comes into play and simplicity wins.

We are moving in a new direction for better or worse and going live. I feel like it is slightly off the right path but sometimes you need to take a wrong step like SOAP did to get back to simple. We'll see how it goes.

agentS11y ago

A binary protocol's parsing is usually something like read 2 bytes from the wire, decode N = uint16(b[0] << 8) | uint16(b[1]), then read N bytes from the wire. A text-based protocol's parsing almost always involves a streaming parser, which is tricky to get correct, and always more inefficient.

Besides, I think this is a moot point, because chances are that less than 100 people's HTTP2 implementations will serve 99.9999% of traffic. It's not like you or I spend much of our time deep in nginx's code debugging some HTTP parsing; I think its just as unlikely we'll be doing that for HTTP2 parsing.

Also, HTTP2 will always (pretty much) be wrapped in TLS. So its not like you're going to be looking at a plain-text dump of that. You'll be using a tool and that tool author will implement a way to convert the binary framing to human-readable text.

Another way to put it is that the vast majority of HTTP streams are not examined by humans and only examined by computers. Choosing a text-based protocol just seems a way to adversely impact the performance of every single user's web-browsing.

Another another way to put it is that there is a reason that Thrift, Protocol Buffers, and other common RPC mechanisms do not use a text-based protocol. Nor do IP, TCP, or UDP, for that matter. And there's a reason that Memcached was updated with a binary protocol even though it had a perfectly serviceable text-based protocol.

1 more reply

MichaelGG11y ago

Binary protocols are usually far easier to implement both sending and receiving. There is far less ambiguity.

In fact, the newline problem I mentioned? It was not easier to diagnose, and was only caught by using tools checking it as a binary structure.

Postel was just flat wrong, and history shows us this is so. JSON is popular because it was automatically available in JavaScript, and people dislike the bit if extra verbosity XML introduces. JSON is also a much tighter format than the text parsing the IETF usually implements.

Postel's law also goes against the idea of failing fast. Instead, you end up thinking you're compliant, because implementations just happen to interpret your mistake in the right way. Then one day something changes and bam, it all comes crashing down. Ask any experienced web developer the weird edge cases they have to deal with, again from Postel's law.

And anyways, you know what everyone uses when debugging the wire? WireShark or something similar. Problem solved. Same for things like JSON. Over the past months I've been dealing with that a lot. Every time I have an issue, I pull out a tool to figure it out.

Do you know the real reason for the text looseness? It's a holdover from the '60s. The Header: Value format was just a slight codification of what people were already doing. And why? Because they wanted a format that was primary written and readby humans, with a bit if structure thrown in. Loose syntax is great in such cases. Modern protocols are not written and rarely read by humans. So it's just a waste of electricity and developer time.

j / k navigate · click thread line to collapse

0 comments

MichaelGG11y ago

drawkboxOP11y ago

Going binary does not make it suddenly easier, it just slices it up and adds a layer of obfuscation.

agentS11y ago

1 more reply

MichaelGG11y ago

Binary protocols are usually far easier to implement both sending and receiving. There is far less ambiguity.

In fact, the newline problem I mentioned? It was not easier to diagnose, and was only caught by using tools checking it as a binary structure.

j / k navigate · click thread line to collapse