undefined | Better HN

0 pointsr3trohack3r11mo ago0 comments

This is one of the key insights in my early SRE career that changed how I viewed software engineering at scale.

Components aren’t reliable. The whole thing might be duct tape and popsicle sticks. But the trick for SRE work is to create stability from unreliable components by isolating and routing around failures.

It’s part of what made chaos engineering so effective. From randomly slowing down disk/network speed to unplugging server racks to making entire datacenters go dark - you intentionally introduce all sorts of crazy failure modes to intentionally break things and make sure the system remains metastable.

0 comments

dbetteridge11mo ago

Everything is chaos, seek not to control it or you will lose your mind.

Seek only to understand it well enough to harness the chaos for more subtle useful purpose, for from chaos comes all the beauty and life in the universe.

ForOldHack11mo ago

Message on a mug: "if carpenters built houses the way programmers write software, a woodpecker could destroy civilization."

The syncronasation of a power grid ... Wow.

vkou11mo ago

If houses could be torn down and recreated with the press of a key, we probably wouldn't have a housing shortage.

alpaca12811mo ago

We would instead have HaaS, with monthly subscriptions for a license to use the house. Which can be randomly revoked at any moment if the company doesn't feel like supporting it is profitable enough, or if an AI thinks your electricity usage is suspicious and permabans you from using a home in the entire town.

2 more replies

ec2y11mo ago

Ha. I live in Australia and unfortunately that’s already about the quality level for new builds.

j / k navigate · click thread line to collapse