The problem isn't the CPU, the problem is the OS going out for lunch, and then designs that assume the OS will go our for lunch (i.e. deep buffers everywhere)... The CPU can move data between devices with sub-microsecond latencies...