When you're just starting the company? When you get your first venture round? When you hire your 10th person?
I know every company I've worked at that didn't have SSO had problems offboarding people because each group (eg, Finance, Product, Engineering, etc.) purchased their own set of SaaS services and there was no central place for IT to manage access.
The downside is cost. Some companies charge 100%+ the base price for sso (https://sso.tax/).
