Nor do I. I make a lot of progress down in the 3% code. You have to go file by file and module by module rather than hotspot by hotspot down here, and you have to be very good at refactoring and writing unit/pinning tests to get away with it. But the place I worked where I developed a lot of my theories I kept this up for over two years, making significant perf improvements every quarter before I had to start looking under previously visited rocks. It also taught me the importance of telegraphing competence to your customers. They will put up with a lot of problems if they trust you to solve them soon.

No I’m saying the jitter is a lot more than one percent, and rounding errors and externalities can be multiples of the jitter. Especially for leaf node functions - which are a fruitful target because it turns out your peers mostly don’t have opinions about “clever” optimizations if they’re in leaf node functions, in commits they could always roll back if they get sick of looking at them. Which is both a technical and a social problem.

One of the seminal cases for my leaving the conventional wisdom: I’d clocked a function call as 10% of an expensive route, via the profiler. I noticed that the call count was way off. A little more than twice what I would have guessed. Turned out two functions were making the call with the same input, but they weren’t far apart in the call tree, so I rearranged the code a bit to eliminate the second call, expecting the e2e time to be 5% faster. But it was repeatably 20% faster, which made no sense at all, except for memory pressure.

About five years later I saw the most egregious case of this during a teachable moment when I was trying to explain to coworkers that fixing dumb code >> caching. I eliminated a duplicate database request in sibling function calls, and we saw the entire response time drop by 10x instead of the 2-3x I was expecting.