UInt32Max (i.e. 4294967295) is divisible by 3, so your code actually is perfectly random (or more accurately, as random as go's rand package). It would be biased with N=4, for example.
Regardless, with small values of N, the bias is very slight so you would need many many iterations to see the imperfection in a statically significant way.