An ideal integrator has a response of 1/s. That's just a 1st order low-pass filter with the pole at 0. Therefore, it will filter out high frequency noise.

> Take a step to the left for heads and a step to the right for tails. Most of the time you won't end up where you started, i.e. you have residual error.

I wrote a quick simulation based on your suggestion [1]. Started by generating 1e6 random points and then applied a high-pass filter. Calculated the cumulative sum on both the original and the filtered version. TL;DR: filtered version has small and very fast variations but doesn't feature the much larger amplitude swings seen in the original.

Integration indeed does not help for those large slow swings (I'd call it drift in case of a gyroscope), but that's what I was trying to get at when I started to distinguish between short and long term random effects. What I was trying to get across originally is that "all the little errors" (which I read to mean tiny fast variations, forgetting that drift is a much bigger issue in gyroscopes) which OP mentioned get filtered/canceled out. I totally missed to explain that this will vary with frequency, which was my bad.

[1] https://github.com/afonsotrepa/noise-sim/tree/master