It's not the pulls that are the problem, it's caching intermediate layers from the build that is the problem. As soon as you introduce a networked registry, the time it takes to pull layers from the registry cache and push them back to the registry cache are frequently not much better than simply rebuilding the layers, not to mention the additional compute/storage cost of running the registry cache itself.

It's just a problem that requires big, local disks to solve.