Eh. I think people drastically underestimate the variance on calories absorbed (the effective amount in) and the calories used for general homeostasis and metabolism (the effective amount out) excluding exercise.
It does indeed have a large effect, the science is clear. Everything from gut bacteria to how often you eat meals can affect the % of caloric intake that gets converted into fat.