undefined | Better HN

0 pointsseoaeu3y ago0 comments

Wait, the proposed solution to a data format being slow to parse is to work around the bad performance by caching the already parsed representation? That seems like it has a clear flaw if you’re only accessing the data once…

0 comments

TuringTest3y ago

Where's the flaw in that? If you're only accessing the data once, why does it matter how fast or slow it is?

And, you're suggesting that user-facing data should be harder to work with only to make it faster to parse?

seoaeuOP3y ago

Accessing any given data once. When you have a total dataset size in the 10-100s of gigabytes range, having to download any significant fraction of it to do data processing is really unfortunate.

But seriously what's up with this total disdain for anyone trying to build applications with OSM data? You don't seem to care whether parsing is near instant or as other commenters have mentioned, literally a majority of total processing time for certain compute jobs

j / k navigate · click thread line to collapse

0 comments

TuringTest3y ago

Where's the flaw in that? If you're only accessing the data once, why does it matter how fast or slow it is?

And, you're suggesting that user-facing data should be harder to work with only to make it faster to parse?

seoaeuOP3y ago

Accessing any given data once. When you have a total dataset size in the 10-100s of gigabytes range, having to download any significant fraction of it to do data processing is really unfortunate.

j / k navigate · click thread line to collapse