No server to manage is the least important part of it all. It should describe the abstraction that's actually being offered: IaaS is servers, PaaS is a single app, FaaS is individual function, SaaS is just software.
Any of those other than IaaS itself can be "serverless".