Longer buffers allow for jitter in processing further upstream. If you get a buffer every 0.6ms, you need to be able to process it always within 0.6ms.