>The point of the anti-caking agent is to prevent the cheese from sticking to itself, which inherently affects the texture of the cheese in your mouth
It does so by keeping the cheese 'dryer' than it normally would be. Putting it in your mouth basically undoes that. You're only going to notice if you're eating it by the handful, not when you're using it in actual food dishes.
