The determining factors driving a computer program can be fully quanitified; the sets of inputs and conditions is finite, can be reasoned over, and described fully.
That's basically the fundamental description of computing, in fact, and what makes a Turing machine.
The determining factors "IRL" are effectively infinite, a causal "chain" of infinite (or near infinite) complexity that expands backwards to the Big Bang, (or whatever) and sideways around the planet and beyond. There is no catalog you could make of all the "causes" that could isolate things enough to truly reason over and describe them all.
And so, yeah, to say it's all just "little programs" is the most ridiculous reductionism, that basically purposefully neglects to see the complexity and depth of the world around us.
I personally take a strongly determinist, materialist philosophical position. But I would never ever express that in terms of "programs" or anything similar.
