undefined | Better HN

0 pointsxwolfi3y ago0 comments

It's a hassle to change shape in a db possibly, but have you lived through changing the shape of data in a json store where historical data is important ?

You either dont care abt the past and cant read it anymore, version your writer and reader each time you realize an address in a new country has yet another frigging field, or parse each json value to add the new shape in place in your store.

Json doesnt solve the problem of shape evolution, but it tempt you very strongly to think you can ignore it.

0 comments

VectorLock3y ago

You do database migrations or you handle data with a variable shape. Where do you want to put your effort? The latter makes rollbacks easier at least.

xwolfiOP3y ago

But you dont care as much about rollback (we rarely rollack successful migration and only on prod issues the next few days, and always prepare for it with a reverse script) as you care about the past (can you read data from 2 years ago? this can matter a lot more, and you must know it's never done on unstructured data: the code evolve with the new shape, week after week and you re cucked when it's the old consumer code you need to unearth to understand the past). It's never perfect, but the belief data dont need structure nor a well automated history of migration, is dangerous.

I've seen myself, anecdotically, that most of the time with json data, the past is ignored during the ramp up to scale and then once the product is alive and popular, problems start arising. Usually the cry is "why the hell dont we have a dumb db rather than this magic dynamic crap". I now work in a more serious giant corporation where dumb dbs are the default and it's way more comfortable than I was led to believe, when I was younger.

j / k navigate · click thread line to collapse