That's not really the issue people have with it.
The issue I have is that it provides a convenient way for people to hassle me for my identity. Being able to identify myself is not a problem I've ever had and I don't like the idea of being forced to buy a smartphone just to remain an employed citizen in the UK.