It's not about being surprised, it's about not reading the dialogue at all.
I make a barcode scanning app in the browser. Our users know that the app scans barcodes using the camera, they receive training on the app before using it, and yet about ~15%~ (Edit: it's closer to 8% now) of our users click "no" when asked to allow camera permissions to scan barcodes. It was causing so many support calls that we ended up doing a 2-stage permissions prompt. First we display a javascript dialog telling the user that they need to click "yes" for the app to work, then we show the native permissions request after they have clicked yes, because if the user selects "no" the first time, we can't ask again, and 99% of users don't know how to go into the settings and re-enable it (hell, even I don't know where it is from memory, and would need to hunt around to find where it is)
And it doesn't create an incentive not to ask at all. At worst the website is where they would be if they didn't ask, at best they are able to display notifications, play audio, play video, get location, etc...
It's a no-lose situation in 99% of cases for the website owner, unless they actually really need that permission to function, then they have a LOT to lose. It's a situation where the incentives are misaligned terribly.
And don't get me wrong, i'm also very VERY happy that I can prevent notifications or location access, but I also think that more could be done to prevent spamming of the permissions prompt itself.