Indeed. Control loops are generally limited by the bandwidth of the system they are controlling, and lag in the system limits the bandwidth you can achieve. Trying to get a PID loop to move a system faster than that system's bandwidth will just create oscillations, and the more tightly the PID is tuned the worse its behaviour when exposed to shocks (especially when the system becomes significantly non-linear in the process).

You can extract more performance out of a system by having a very good predictive model of it and measuring its inputs: This can really improve how you drive the system but it's much more difficult to achieve, and you're still limited by how quickly information moves through the system in terms of how you can react.