It's slow because the original telop is slow, and the learned controllers through imitation learning is always a bit slower.
Source : i work on this (not at PI)
Pi0 uses ARX robot arm which weights 3-4kg per arm. It can easily break things or harm people if you allow it to move fast.
But yeah in general the physical world is more dangerous than we tend to think
