It was also the awful amount of indirection of X(Free86/org) which made sure you had to jump through hoops to get anything on the screen. Even with accelerated hardware X doesn't feel as fast as Windows/macOS due to the insane amount of round-trips for perceived 'network transparency' which doesn't work that great and almost nobody uses.
I find it very disappointing some people are still fighting Wayland which, while not perfect, at least tries to get Linux desktops graphics stack 'on par' with macOS versions from 20 years ago...